A random matrix analysis of random Fourier features: beyond the Gaussian kernel, a precise phase transition, and the corresponding double descent*

DOI10.1088/1742-5468/AC3A77OpenAlexW3098901286MaRDI QIDQ5020045FDOQ5020045

Authors: Zhenyu Liao, R. Couillet, Michael W. Mahoney 

Publication date: 3 January 2022

Published in: Journal of Statistical Mechanics: Theory and Experiment (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2006.05013

Recommendations

• Conditioning of random Fourier feature matrices: double descent and generalization error
• Towards a unified analysis of random Fourier features
• The Generalization Error of Random Features Regression: Precise Asymptotics and the Double Descent Curve
• scientific article; zbMATH DE number 7255176

zbMATH Keywords

deep learninglearning theoryanalysis of algorithmsrandom matrix theory and extensions

Mathematics Subject Classification ID

Statistical mechanics, structure of matter (82-XX)

Cites Work

• The elements of statistical learning. Data mining, inference, and prediction
• Title not available (Why is that?)
• On the equivalence between kernel quadrature rules and random feature expansions
• Random matrix methods for wireless communications.
• Deterministic equivalents for certain functionals of large random matrices
• Information, Physics, and Computation
• Statistical mechanics of learning
• Rigorous learning curve bounds from statistical mechanics
• Just interpolate: kernel ``ridgeless regression can generalize
• Reconciling modern machine-learning practice and the classical bias-variance trade-off
• High-dimensional dynamics of generalization error in neural networks
• A random matrix approach to neural networks
• High-dimensional asymptotics of prediction: ridge regression and classification
• Benign overfitting in linear regression
• The Generalization Error of Random Features Regression: Precise Asymptotics and the Double Descent Curve
• A model of double descent for high-dimensional binary linear classification
• On Random Matrices Arising in Deep Neural Networks. Gaussian Case

Cited In (6)

• Binary classification of Gaussian mixtures: abundance of support vectors, benign overfitting, and regularization
• Precise learning curves and higher-order scaling limits for dot-product kernel regression
• Benign Overfitting and Noisy Features
• Title not available (Why is that?)
• Conditioning of random Fourier feature matrices: double descent and generalization error
• Breaking the waves: asymmetric random periodic features for low-bitrate kernel machines

Uses Software

• MNIST
• Fashion-MNIST

This page was built for publication: A random matrix analysis of random Fourier features: beyond the Gaussian kernel, a precise phase transition, and the corresponding double descent*

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5020045)