Kernel ridge vs. principal component regression: minimax bounds and the qualification of regularization operators
From MaRDI portal
(Redirected from Publication:521337)
Abstract: Regularization is an essential element of virtually all kernel methods for nonparametric regression problems. A critical factor in the effectiveness of a given kernel method is the type of regularization that is employed. This article compares and contrasts members from a general class of regularization techniques, which notably includes ridge regression and principal component regression. We derive an explicit finite-sample risk bound for regularization-based estimators that simultaneously accounts for (i) the structure of the ambient function space, (ii) the regularity of the true regression function, and (iii) the adaptability (or qualification) of the regularization. A simple consequence of this upper bound is that the risk of the regularization-based estimators matches the minimax rate in a variety of settings. The general bound also illustrates how some regularization techniques are more adaptable than others to favorable regularity properties that the true regression function may possess. This, in particular, demonstrates a striking difference between kernel ridge regression and kernel principal component regression. Our theoretical results are supported by numerical experiments.
Recommendations
- Kernel regression, minimax rates and effective dimensionality: beyond the regular case
- A risk comparison of ordinary least squares vs ridge regression
- On the improved rates of convergence for Matérn-type kernel ridge regression with application to calibration of computer models
- Kernel ridge regression
- Model selection in kernel ridge regression
Cited in
(27)- scientific article; zbMATH DE number 7370593 (Why is no real title available?)
- On principal components regression, random projections, and column subsampling
- Kernel ridge regression
- Kernel regression, minimax rates and effective dimensionality: beyond the regular case
- A risk comparison of ordinary least squares vs ridge regression
- Spectrally-truncated kernel ridge regression and its free lunch
- On the improved rates of convergence for Matérn-type kernel ridge regression with application to calibration of computer models
- Optimal rates for spectral algorithms with least-squares regression over Hilbert spaces
- Parallelizing spectrally regularized kernel algorithms
- Weighted spectral filters for kernel interpolation on spheres: estimates of prediction accuracy for noisy data
- Sparse principal component regression via singular value decomposition approach
- Thresholded spectral algorithms for sparse approximations
- Functional principal subspace sampling for large scale functional data analysis
- Spectral algorithms for learning with dependent observations
- Distributed kernel-based gradient descent algorithms
- scientific article; zbMATH DE number 7306853 (Why is no real title available?)
- Optimal rates for multi-pass stochastic gradient methods
- Kernel partial least squares for stationary data
- On the predictive potential of kernel principal components
- Distributed kernel ridge regression with communications
- Sobolev norm learning rates for regularized least-squares algorithms
- Nonasymptotic analysis of robust regression with modified Huber's loss
- A Comparative Study of Pairwise Learning Methods Based on Kernel Ridge Regression
- Randomized estimation of functional covariance operator via subsampling
- Kernel conjugate gradient methods with random projections
- Convergences of regularized algorithms and stochastic gradient methods with random projections
- Some equivalence relationships of regularized regressions
This page was built for publication: Kernel ridge vs. principal component regression: minimax bounds and the qualification of regularization operators
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q521337)
