Convergence rates of kernel conjugate gradient for random design regression
From MaRDI portal
Publication:2835985
Abstract: We prove statistical rates of convergence for kernel-based least squares regression from i.i.d. data using a conjugate gradient algorithm, where regularization against overfitting is obtained by early stopping. This method is related to Kernel Partial Least Squares, a regression method that combines supervised dimensionality reduction with least squares projection. Following the setting introduced in earlier related literature, we study so-called "fast convergence rates" depending on the regularity of the target regression function (measured by a source condition in terms of the kernel integral operator) and on the effective dimensionality of the data mapped into the kernel space. We obtain upper bounds, essentially matching known minimax lower bounds, for the (prediction) norm as well as for the stronger Hilbert norm, if the true regression function belongs to the reproducing kernel Hilbert space. If the latter assumption is not fulfilled, we obtain similar convergence rates for appropriate norms, provided additional unlabeled data are available.
Recommendations
- Conjugate gradients for kernel machines
- Kernel regression, minimax rates and effective dimensionality: beyond the regular case
- Convergences of regularized algorithms and stochastic gradient methods with random projections
- Fast learning rates for regularized regression algorithms
- Learning rates of least-square regularized regression
Cites work
- scientific article; zbMATH DE number 783550 (Why is no real title available?)
- scientific article; zbMATH DE number 936298 (Why is no real title available?)
- 10.1162/15324430260185556
- An introduction to support vector machines and other kernel-based learning methods.
- Cross-validation based adaptation for regularization operators in learning theory
- Learning from examples as an inverse problem
- Optimal rates for the regularized least-squares algorithm
- The Collinearity Problem in Linear Regression. The Partial Least Squares (PLS) Approach to Generalized Inverses
Cited in
(30)- Analysis of regularized Nyström subsampling for regression functions of low smoothness
- Toward Efficient Ensemble Learning with Structure Constraints: Convergent Algorithms and Applications
- Optimal learning rates for distribution regression
- Convergences of regularized algorithms and stochastic gradient methods with random projections
- Distributed learning with indefinite kernels
- On the improved rates of convergence for Matérn-type kernel ridge regression with application to calibration of computer models
- Adaptive parameter selection for kernel ridge regression
- Distributed minimum error entropy algorithms
- Distributed kernel ridge regression with communications
- Kernel regression, minimax rates and effective dimensionality: beyond the regular case
- Kernel conjugate gradient methods with random projections
- Compact convex projections
- scientific article; zbMATH DE number 7370593 (Why is no real title available?)
- On the convergence rate of sparse grid least squares regression
- Error analysis of the kernel regularized regression based on refined convex losses and RKBSs
- Conjugate gradients for kernel machines
- On a regularization of unsupervised domain adaptation in RKHS
- Analysis of target data-dependent greedy kernel algorithms: convergence rates for \(f\)-, \(f \cdot P\)- and \(f/P\)-greedy
- From inexact optimization to learning via gradient concentration
- Asymptotic analysis for affine point processes with large initial intensity
- Distributed learning with regularized least squares
- Faster kernel ridge regression using sketching and preconditioning
- Boosted kernel ridge regression: optimal learning rates and early stopping
- Optimal rates for regularization of statistical inverse learning problems
- Convergence analysis for kernel-regularized online regression associated with an RRKHS
- Distributed least squares prediction for functional linear regression*
- Learning rates for the kernel regularized regression with a differentiable strongly convex loss
- Semi-supervised learning with summary statistics
- Capacity dependent analysis for functional online learning algorithms
- Accelerate stochastic subgradient method by leveraging local growth condition
This page was built for publication: Convergence rates of kernel conjugate gradient for random design regression
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2835985)
