When do neural networks outperform kernel methods?*

DOI10.1088/1742-5468/AC3A81MaRDI QIDQ5020050zbMATH OpenOpenAlexFDO

Authors

Publication date 3 January 2022

Published in Journal of Statistical Mechanics: Theory and Experiment (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2006.13409

zbMATH Keywords

deep learninglearning theorymachine learning

Mathematics Subject Classification ID

Statistical mechanics, structure of matter (82-XX)

Recommendations

• Linearized two-layers neural networks in high dimension
• Kolmogorov width decay and poor approximators in machine learning: shallow neural networks, random feature models and neural tangent kernels
• Convolutional spectral kernel learning with generalization guarantees
• Training Neural Networks as Learning Data-adaptive Kernels: Provable Representation and Approximation Benefits

Cites work

• A mean field view of the landscape of two-layer neural networks
• Adapting to Unknown Smoothness via Wavelet Shrinkage
• Breaking the curse of dimensionality with convex neural networks
• The landscape of empirical risk for nonconvex losses

Cited in

(19)

• The interpolation phase transition in neural networks: memorization and generalization under lazy training
• Generalization error of random feature and kernel methods: hypercontractivity and kernel matrix concentration
• On some similarities and differences between deep neural networks and kernel learning machines
• Learning curves of generic features maps for realistic datasets with a teacher-student model*
• Generalization error rates in kernel regression: the crossover from the noiseless to noisy regime*
• Particle dual averaging: optimization of mean field neural network with global convergence rate analysis*
• Locality defeats the curse of dimensionality in convolutional teacher–student scenarios*
• Relative stability toward diffeomorphisms indicates performance in deep nets*
• Deep learning: a statistical viewpoint
• Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation
• Algorithmic Regularization in Model-Free Overparametrized Asymmetric Matrix Factorization
• When Do Neural Networks Outperform Kernel Methods?
• Learning sparse features can lead to overfitting in neural networks
• Conservative SPDEs as fluctuating mean field limits of stochastic gradient descent
• Modeling structured data learning with restricted Boltzmann machines in the teacher-student setting
• Random features and polynomial rules
• Deterministic equivalent and error universality of deep random features learning
• Bayes-optimal learning of deep random networks of extensive-width
• A Lipschitz spaces view of infinitely wide shallow neural networks

Describes a project that uses

Uses Software

• TensorFlow
• neural-tangents
• Adam

This page was built for publication: When do neural networks outperform kernel methods?*

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5020050)