Convergence rates of kernel conjugate gradient for random design regression
From MaRDI portal
(Redirected from Publication:2835985)
Abstract: We prove statistical rates of convergence for kernel-based least squares regression from i.i.d. data using a conjugate gradient algorithm, where regularization against overfitting is obtained by early stopping. This method is related to Kernel Partial Least Squares, a regression method that combines supervised dimensionality reduction with least squares projection. Following the setting introduced in earlier related literature, we study so-called "fast convergence rates" depending on the regularity of the target regression function (measured by a source condition in terms of the kernel integral operator) and on the effective dimensionality of the data mapped into the kernel space. We obtain upper bounds, essentially matching known minimax lower bounds, for the (prediction) norm as well as for the stronger Hilbert norm, if the true regression function belongs to the reproducing kernel Hilbert space. If the latter assumption is not fulfilled, we obtain similar convergence rates for appropriate norms, provided additional unlabeled data are available.
Recommendations
- Conjugate gradients for kernel machines
- Kernel regression, minimax rates and effective dimensionality: beyond the regular case
- Convergences of regularized algorithms and stochastic gradient methods with random projections
- Fast learning rates for regularized regression algorithms
- Learning rates of least-square regularized regression
Cites work
- scientific article; zbMATH DE number 783550 (Why is no real title available?)
- scientific article; zbMATH DE number 936298 (Why is no real title available?)
- 10.1162/15324430260185556
- An introduction to support vector machines and other kernel-based learning methods.
- Cross-validation based adaptation for regularization operators in learning theory
- Learning from examples as an inverse problem
- Optimal rates for the regularized least-squares algorithm
- The Collinearity Problem in Linear Regression. The Partial Least Squares (PLS) Approach to Generalized Inverses
Cited in
(34)- Error analysis of the kernel regularized regression based on refined convex losses and RKBSs
- scientific article; zbMATH DE number 7370593 (Why is no real title available?)
- Distributed learning with indefinite kernels
- Kernel regression, minimax rates and effective dimensionality: beyond the regular case
- On the improved rates of convergence for Matérn-type kernel ridge regression with application to calibration of computer models
- Learning rates for the kernel regularized regression with a differentiable strongly convex loss
- Distributed learning with regularized least squares
- Semi-supervised learning with summary statistics
- Capacity dependent analysis for functional online learning algorithms
- Distributed least squares prediction for functional linear regression*
- Conjugate gradients for kernel machines
- Distributed minimum error entropy algorithms
- Optimal rates for regularization of statistical inverse learning problems
- Distributed kernel ridge regression with communications
- The impact of smoothness of kernels and target functions on unsupervised covariate shift adaptation in RKHS
- Convergence analysis for kernel-regularized online regression associated with an RRKHS
- From inexact optimization to learning via gradient concentration
- Comparing regularisation paths of (conjugate) gradient estimators in ridge regression
- Compact convex projections
- Analysis of regularized Nyström subsampling for regression functions of low smoothness
- Early stopping for conjugate gradients in statistical inverse problems
- Adaptive distributed kernel ridge regression: a feasible distributed learning scheme for data silos
- Adaptive parameter selection for kernel ridge regression
- On a regularization of unsupervised domain adaptation in RKHS
- Boosted kernel ridge regression: optimal learning rates and early stopping
- Asymptotic analysis for affine point processes with large initial intensity
- Optimal learning rates for distribution regression
- Analysis of target data-dependent greedy kernel algorithms: convergence rates for \(f\)-, \(f \cdot P\)- and \(f/P\)-greedy
- Toward Efficient Ensemble Learning with Structure Constraints: Convergent Algorithms and Applications
- Faster kernel ridge regression using sketching and preconditioning
- Kernel conjugate gradient methods with random projections
- Accelerate stochastic subgradient method by leveraging local growth condition
- Convergences of regularized algorithms and stochastic gradient methods with random projections
- On the convergence rate of sparse grid least squares regression
This page was built for publication: Convergence rates of kernel conjugate gradient for random design regression
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2835985)
