Deep learning: a statistical viewpoint

DOI10.1017/S0962492921000027OpenAlexW3191067499MaRDI QIDQ5887827

No author found.

Publication date: 14 April 2023

Published in: Acta Numerica (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2103.09177

Mathematics Subject Classification ID

Fluid mechanics (76-XX) Numerical analysis (65-XX)

Related Items

Surprises in high-dimensional ridgeless least squares interpolation ⋮ Optimal regularizations for data generation with probabilistic graphical models ⋮ A note on the prediction error of principal component regression in high dimensions ⋮ Adversarial examples in random neural networks with general activations ⋮ Free dynamics of feature learning processes ⋮ Mini-workshop: Mathematical foundations of robust and generalizable learning. Abstracts from the mini-workshop held October 2--8, 2022 ⋮ Should we estimate a product of density functions by a product of estimators? ⋮ High-Dimensional Analysis of Double Descent for Linear Regression with Random Projections ⋮ Measuring Complexity of Learning Schemes Using Hessian-Schatten Total Variation ⋮ Tractability from overparametrization: the example of the negative perceptron ⋮ The interpolation phase transition in neural networks: memorization and generalization under lazy training

Uses Software

• ElemStatLearn

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Greedy function approximation: A gradient boosting machine.
• Concentration inequalities and moment bounds for sample covariance operators
• Empirical entropy, minimax regret and minimax risk
• Polynomial bounds for VC dimension of sigmoidal and general Pfaffian neural networks
• Anisotropic local laws for random matrices
• The spectrum of kernel random matrices
• Kernels as features: on kernels, margins, and low-dimensional mappings
• Combinatorics of random processes and sections of convex bodies
• Estimating a regression function
• Decision theoretic generalizations of the PAC model for neural net and other learning applications
• The densest hemisphere problem
• The Hilbert kernel regression estimate.
• Sharper bounds for Gaussian and empirical processes
• A decision-theoretic generalization of on-line learning and an application to boosting
• Hardness results for neural network approximation problems
• A random matrix approach to neural networks
• The spectral norm of random inner-product kernel matrices
• A note on margin-based loss functions in classification
• Arcing classifiers. (With discussion)
• Boosting the margin: a new explanation for the effectiveness of voting methods
• A general lower bound on the number of examples needed for learning
• On the Bayes-risk consistency of regularized boosting methods.
• Statistical behavior and consistency of classification methods based on convex risk minimization.
• Support-vector networks
• An inequality for uniform deviations of sample averages from their means
• Linearized two-layers neural networks in high dimension
• The interpolation phase transition in neural networks: memorization and generalization under lazy training
• Surprises in high-dimensional ridgeless least squares interpolation
• Generalization error of random feature and kernel methods: hypercontractivity and kernel matrix concentration
• Products of many large random matrices and gradients in deep neural networks
• Gradient descent optimizes over-parameterized deep ReLU networks
• Just interpolate: kernel ``ridgeless regression can generalize
• Optimal transport for applied mathematicians. Calculus of variations, PDEs, and modeling
• Local Rademacher complexities and oracle inequalities in risk minimization. (2004 IMS Medallion Lecture). (With discussions and rejoinder)
• Optimal rates for the regularized least-squares algorithm
• Theoretical foundations of the potential function method in pattern recognition learning
• Local Rademacher complexities
• Boosting with early stopping: convergence and consistency
• THE SPECTRUM OF RANDOM KERNEL MATRICES: UNIVERSALITY RESULTS FOR ROUGH AND VARYING KERNELS
• THE SPECTRUM OF RANDOM INNER-PRODUCT KERNEL MATRICES
• Bounding the Smallest Singular Value of a Random Matrix Without Concentration
• Learnability and the Vapnik-Chervonenkis dimension
• FAST RATES FOR ESTIMATION ERROR AND ORACLE INEQUALITIES FOR MODEL SELECTION
• Distribution-free inequalities for the deleted and holdout error estimates
• Efficient agnostic learning of neural networks with bounded fan-in
• The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network
• Rademacher penalties and structural risk minimization
• Bounds on rates of variable-basis and neural-network approximation
• Comparison of worst case errors in linear and neural network approximation
• High-Dimensional Probability
• Improving the sample complexity using global data
• 10.1162/153244302760200704
• 10.1162/153244303321897690
• Analysis of Two Simple Heuristics on a Random Instance ofk-sat
• Neural Network Learning
• A mean field view of the landscape of two-layer neural networks
• Size-independent sample complexity of neural networks
• When do neural networks outperform kernel methods?*
• The Generalization Error of Random Features Regression: Precise Asymptotics and the Double Descent Curve
• Benign overfitting in linear regression
• Extending the scope of the small-ball method
• Does learning require memorization? a short tale about a long tail
• Reconciling modern machine-learning practice and the classical bias–variance trade-off
• Mean Field Analysis of Neural Networks: A Law of Large Numbers
• Breaking the Curse of Dimensionality with Convex Neural Networks
• A Better Algorithm for Random k-SAT
• Nearest neighbor pattern classification
• On the Uniform Convergence of Relative Frequencies of Events to Their Probabilities
• Convexity, Classification, and Risk Bounds
• Nonlinear random matrix theory for deep learning
• A jamming transition from under- to over-parametrization affects generalization in deep learning
• Introduction to nonparametric estimation
• The elements of statistical learning. Data mining, inference, and prediction
• Model selection and error estimation