High-Dimensional Analysis of Double Descent for Linear Regression with Random Projections
From MaRDI portal
Publication:6151666
Abstract: We consider linear regression problems with a varying number of random projections, where we provably exhibit a double descent curve for a fixed prediction problem, with a high-dimensional analysis based on random matrix theory. We first consider the ridge regression estimator and review earlier results using classical notions from non-parametric statistics, namely degrees of freedom, also known as effective dimensionality. We then compute asymptotic equivalents of the generalization performance (in terms of squared bias and variance) of the minimum norm least-squares fit with random projections, providing simple expressions for the double descent phenomenon.
Cites work
- A statistical perspective on randomized sketching for ordinary least-squares
- An elementary analysis of ridge regression with random design
- Benign overfitting in linear regression
- Deep learning: a statistical viewpoint
- Distributed linear regression by averaging
- Eigenvectors of some large sample covariance matrix ensembles
- Hanson-Wright inequality and sub-Gaussian concentration
- High-dimensional asymptotics of prediction: ridge regression and classification
- scientific article; zbMATH DE number 47282 (Why is no real title available?)
- Learning Bounds for Kernel Regression Using Effective Data Dimensionality
- On the empirical distribution of eigenvalues of a class of large dimensional random matrices
- Optimal rates for the regularized least-squares algorithm
- Reconciling modern machine-learning practice and the classical bias-variance trade-off
- Ridge Regression: Biased Estimation for Nonorthogonal Problems
- Scaling description of generalization with number of parameters in deep learning
- Spectral analysis of large dimensional random matrices
- Spectral convergence for a general class of random matrices
- Strong convergence of the empirical distribution of eigenvalues of large dimensional random matrices
- Surprises in high-dimensional ridgeless least squares interpolation
- The Generalization Error of Random Features Regression: Precise Asymptotics and the Double Descent Curve
- The interpolation phase transition in neural networks: memorization and generalization under lazy training
- Two models of double descent for weak features
- WONDER: weighted one-shot distributed ridge regression in high dimensions
Cited in
(3)
This page was built for publication: High-Dimensional Analysis of Double Descent for Linear Regression with Random Projections
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6151666)
