Online Pairwise Learning Algorithms
From MaRDI portal
Abstract: Pairwise learning usually refers to a learning task which involves a loss function depending on pairs of examples, among which most notable ones include ranking, metric learning and AUC maximization. In this paper, we study an online algorithm for pairwise learning with a least-square loss function in an unconstrained setting of a reproducing kernel Hilbert space (RKHS), which we refer to as the Online Pairwise lEaRning Algorithm (OPERA). In contrast to existing works cite{Kar,Wang} which require that the iterates are restricted to a bounded domain or the loss function is strongly-convex, OPERA is associated with a non-strongly convex objective function and learns the target function in an unconstrained RKHS. Specifically, we establish a general theorem which guarantees the almost surely convergence for the last iterate of OPERA without any assumptions on the underlying distribution. Explicit convergence rates are derived under the condition of polynomially decaying step sizes. We also establish an interesting property for a family of widely-used kernels in the setting of pairwise learning and illustrate the above convergence results using such kernels. Our methodology mainly depends on the characterization of RKHSs using its associated integral operators and probability inequalities for random variables with values in a Hilbert space.
Recommendations
- Online pairwise learning algorithms with convex loss functions
- Online learning algorithms
- Online regularized pairwise learning with least squares loss
- Online regularized pairwise learning with non-i.i.d. observations
- Online regularized learning with pairwise loss functions
- scientific article; zbMATH DE number 1569102
- Online gradient descent learning algorithms
- A primal-dual perspective of online learning algorithms
- scientific article; zbMATH DE number 1950582
- On Complexity Issues of Online Learning Algorithms
Cites work
- 10.1162/153244303321897690
- A Stochastic Approximation Method
- An Explicit Description of the Reproducing Kernel Hilbert Spaces of Gaussian RBF Kernels
- Distance metric learning for large margin nearest neighbor classification
- Distance metric learning with eigenvalue optimization
- Estimation of gradients and coordinate covariation in classification
- Generalization bounds for metric and similarity learning
- Generalization bounds for ranking algorithms via algorithmic stability
- scientific article; zbMATH DE number 3536702 (Why is no real title available?)
- Large scale online learning of image similarity through ranking
- Learning coordinate covariances via gradients
- Learning rates of least-square regularized regression
- Learning Theory
- Learning theory estimates via integral operators and their approximations
- Model selection for regularized least-squares algorithm in learning theory
- On early stopping in gradient descent learning
- On ranking and generalization bounds
- On the Generalization Ability of On-Line Learning Algorithms
- On the mathematical foundations of learning
- Online gradient descent learning algorithms
- Online learning algorithms
- Online Learning as Stochastic Approximation of Regularization Paths: Optimality and Almost-Sure Convergence
- Online Regularized Classification Algorithms
- Optimum bounds for the distributions of martingales in Banach spaces
- Ranking and empirical minimization of \(U\)-statistics
- Regularization schemes for minimum error entropy principle
- Robust Stochastic Approximation Approach to Stochastic Programming
- Support Vector Machines
- Theory of Reproducing Kernels
- Universal kernels
- Unregularized online learning algorithms with general loss functions
Cited in
(26)- Robust pairwise learning with Huber loss
- Generalization ability of online pairwise support vector machine
- Distributed kernel gradient descent algorithm for minimum error entropy principle
- Convergence of online pairwise regression learning with quadratic loss
- Stochastic AUC optimization with general loss
- Kernel gradient descent algorithm for information theoretic learning
- Online pairwise learning algorithms with convex loss functions
- Fast and strong convergence of online learning algorithms
- Online regularized learning with pairwise loss functions
- Kernel-based online gradient descent using distributed approach
- Parameter learning algorithm for the online data acknowledgment problem
- On-line learning in parity machines
- Regularized Nyström subsampling in regression and ranking problems under general smoothness assumptions
- Distributed minimum error entropy algorithms
- Stochastic proximal AUC maximization
- Iterative gradient descent for outlier detection
- Distributed spectral pairwise ranking algorithms
- Online regularized pairwise learning with non-i.i.d. observations
- Stability and optimization error of stochastic gradient descent for pairwise learning
- Online regularized pairwise learning with least squares loss
- Convergence analysis of distributed multi-penalty regularized pairwise learning
- Analysis of Online Composite Mirror Descent Algorithm
- Online minimum error entropy algorithm with unbounded sampling
- Convergence analysis for kernel-regularized online regression associated with an RRKHS
- Error analysis of kernel regularized pairwise learning with a strongly convex loss
- Pairwise learning problems with regularization networks and Nyström subsampling approach
This page was built for publication: Online Pairwise Learning Algorithms
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5380417)
