Generalization error of minimum weighted norm and kernel interpolation
From MaRDI portal
Abstract: We study the generalization error of functions that interpolate prescribed data points and are selected by minimizing a weighted norm. Under natural and general conditions, we prove that both the interpolants and their generalization errors converge as the number of parameters grow, and the limiting interpolant belongs to a reproducing kernel Hilbert space. This rigorously establishes an implicit bias of minimum weighted norm interpolation and explains why norm minimization may either benefit or suffer from over-parameterization. As special cases of this theory, we study interpolation by trigonometric polynomials and spherical harmonics. Our approach is from a deterministic and approximation theory viewpoint, as opposed to a statistical or random matrix one.
Recommendations
- Overparameterization and generalization error: weighted trigonometric interpolation
- Just interpolate: kernel ``ridgeless regression can generalize
- Interpolation via weighted \(\ell_{1}\) minimization
- Valid Generalisation from Approximate Interpolation
- Learning with generalization capability by kernel methods of bounded complexity
Cites work
- L^\infty -Multivariate Approximation Theory
- Approximation in Sobolev spaces by kernel expansions
- Benign overfitting in linear regression
- Deep vs. shallow networks: an approximation theory perspective
- scientific article; zbMATH DE number 1950575 (Why is no real title available?)
- Interpolation via weighted \(\ell_{1}\) minimization
- Just interpolate: kernel ``ridgeless regression can generalize
- Local error estimates for radial basis function interpolation of scattered data
- Mercer theorem for RKHS on noncompact sets
- Minimum Sobolev norm interpolation with trigonometric polynomials on the torus
- Modern Fourier Analysis
- On the mathematical foundations of learning
- Recent developments in error estimates for scattered-data interpolation via radial basis functions
- Reconciling modern machine-learning practice and the classical bias-variance trade-off
- Scattered Data Approximation
- Scattered Data Interpolation on Spheres: Error Estimates and Locally Supported Basis Functions
- Scattered-Data Interpolation on \bb R^\protectn: Error Estimates for Radial Basis and Band-Limited Functions
- Spherical harmonics in \(p\) dimensions
- Theory of Reproducing Kernels
Cited in
(7)- On the robustness of minimum norm interpolators and regularized empirical risk minimizers
- Just interpolate: kernel ``ridgeless regression can generalize
- The generalization error of the minimum-norm solutions for over-parameterized neural networks
- Dimensionality Reduction, Regularization, and Generalization in Overparameterized Regressions
- Overparameterization and generalization error: weighted trigonometric interpolation
- Weighted neural tangent kernel: a generalized and improved network-induced kernel
- Multiscale estimates for the condition number of non-harmonic Fourier matrices
This page was built for publication: Generalization error of minimum weighted norm and kernel interpolation
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4999364)
