A random matrix analysis of random Fourier features: beyond the Gaussian kernel, a precise phase transition, and the corresponding double descent*

DOI10.1088/1742-5468/AC3A77MaRDI QIDQ5020045zbMATH OpenOpenAlexFDO

Authors Zhenyu Liao, R. Couillet, Michael W. Mahoney

Publication date 3 January 2022

Published in Journal of Statistical Mechanics: Theory and Experiment (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2006.05013

zbMATH Keywords

deep learninglearning theoryanalysis of algorithmsrandom matrix theory and extensions

Mathematics Subject Classification ID

Statistical mechanics, structure of matter (82-XX)

Recommendations

• Conditioning of random Fourier feature matrices: double descent and generalization error
• Towards a unified analysis of random Fourier features
• The Generalization Error of Random Features Regression: Precise Asymptotics and the Double Descent Curve
• scientific article; zbMATH DE number 7255176

Cites work

• scientific article; zbMATH DE number 1332320 (Why is no real title available?)
• A model of double descent for high-dimensional binary linear classification
• A random matrix approach to neural networks
• Benign overfitting in linear regression
• Deterministic equivalents for certain functionals of large random matrices
• High-dimensional asymptotics of prediction: ridge regression and classification
• High-dimensional dynamics of generalization error in neural networks
• Information, Physics, and Computation
• Just interpolate: kernel ``ridgeless regression can generalize
• On Random Matrices Arising in Deep Neural Networks. Gaussian Case
• On the equivalence between kernel quadrature rules and random feature expansions
• Random matrix methods for wireless communications.
• Reconciling modern machine-learning practice and the classical bias-variance trade-off
• Rigorous learning curve bounds from statistical mechanics
• Statistical mechanics of learning
• The Generalization Error of Random Features Regression: Precise Asymptotics and the Double Descent Curve
• The elements of statistical learning. Data mining, inference, and prediction

Cited in

(6)

• Binary classification of Gaussian mixtures: abundance of support vectors, benign overfitting, and regularization
• Precise learning curves and higher-order scaling limits for dot-product kernel regression
• Benign Overfitting and Noisy Features
• scientific article; zbMATH DE number 7255176 (Why is no real title available?)
• Conditioning of random Fourier feature matrices: double descent and generalization error
• Breaking the waves: asymmetric random periodic features for low-bitrate kernel machines

Describes a project that uses

Uses Software

• Fashion-MNIST
• MNIST

This page was built for publication: A random matrix analysis of random Fourier features: beyond the Gaussian kernel, a precise phase transition, and the corresponding double descent*

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5020045)