A priori estimates of the population risk for two-layer neural networks
From MaRDI portal
Publication:2282333
Abstract: New estimates for the population risk are established for two-layer neural networks. These estimates are nearly optimal in the sense that the error rates scale in the same way as the Monte Carlo error rates. They are equally effective in the over-parametrized regime when the network size is much larger than the size of the dataset. These new estimates are a priori in nature in the sense that the bounds depend only on some norms of the underlying functions to be fitted, not the parameters in the model, in contrast with most existing results which are a posteriori in nature. Using these a priori estimates, we provide a perspective for understanding why two-layer neural networks perform better than the related kernel methods.
Recommendations
- Nonlinear weighted directed acyclic graph and a priori estimates for neural networks
- Linearized two-layers neural networks in high dimension
- A mean field view of the landscape of two-layer neural networks
- A comparative analysis of optimization and generalization properties of two-layer neural network and random feature models under gradient descent dynamics
- Approximation and estimation bounds for artificial neural networks
Cited in
(63)- Optimal approximation rate of ReLU networks in terms of width and depth
- Supervised learning from noisy observations: combining machine-learning techniques with data assimilation
- Representation formulas and pointwise properties for Barron functions
- The Barron space and the flow-induced function spaces for neural network models
- Machine learning for prediction with missing dynamics
- SelectNet: self-paced learning for high-dimensional partial differential equations
- A comparative analysis of optimization and generalization properties of two-layer neural network and random feature models under gradient descent dynamics
- Kolmogorov width decay and poor approximators in machine learning: shallow neural networks, random feature models and neural tangent kernels
- Adaptive deep neural networks methods for high-dimensional partial differential equations
- Multi-scale fusion network: a new deep learning structure for elliptic interface problems
- Low-rank kernel approximation of Lyapunov functions using neural networks
- Enhancing Accuracy of Deep Learning Algorithms by Training with Low-Discrepancy Sequences
- Deep network with approximation error being reciprocal of width to power of square root of depth
- When do extended physics-informed neural networks (XPINNs) improve generalization?
- An Augmented Lagrangian Deep Learning Method for Variational Problems with Essential Boundary Conditions
- A Deep Learning Method for Elliptic Hemivariational Inequalities
- Nonlinear weighted directed acyclic graph and a priori estimates for neural networks
- Deep Network Approximation for Smooth Functions
- Machine learning and computational mathematics
- Finite neuron method and convergence analysis
- Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks
- Deep network approximation characterized by number of neurons
- Molecular modeling by machine learning
- Stationary Density Estimation of Itô Diffusions Using Deep Learning
- Simultaneous neural network approximation for smooth functions
- Piecewise linear functions representable with infinite width shallow ReLU neural networks
- Deep Neural Networks for Solving Large Linear Systems Arising from High-Dimensional Problems
- Neural network approximation: three hidden layers are enough
- A class of dimension-free metrics for the convergence of empirical measures
- A framework for machine learning of model error in dynamical systems
- Overall error analysis for the training of deep neural networks via stochastic gradient descent with random initialisation
- A mathematical perspective of machine learning
- Neural network approximation and estimation of classifiers with classification boundary in a Barron class
- Deep learning via dynamical systems: an approximation perspective
- On mathematical modeling in image reconstruction and beyond
- Solving PDEs on unknown manifolds with machine learning
- Generalization error in the deep Ritz method with smooth activation functions
- Strong overall error analysis for the training of artificial neural networks via random initializations
- DRM revisited: a complete error analysis
- An L^2 analysis of reinforcement learning in high dimensions with kernel and neural network approximation
- Towards a mathematical understanding of neural network-based machine learning: what we know and what we don't
- On the Banach spaces associated with multi-layer ReLU networks: function representation, approximation theory and gradient descent dynamics
- Spectral Barron space for deep neural network approximation
- ReLU neural networks with linear layers are biased towards single- and multi-index models
- A duality framework for analyzing random feature and two-layer neural networks
- A hybrid iterative method for elliptic variational inequalities of the second kind
- Overview frequency principle/spectral bias in deep learning
- Embedding inequalities for Barron-type spaces
- Reinforcement learning with function approximation: from linear to nonlinear
- A brief survey on the approximation theory for sequence modelling
- Approximation of functionals by neural network without curse of dimensionality
- A priori estimates for deep residual network in continuous-time reinforcement learning
- Duality for neural networks through reproducing kernel Banach spaces
- Convergence analysis of PINNs with over-parameterization
- Convergence rates for random feature neural network approximation in molecular dynamics
- Elliptic Regularity Theory in Barron Spaces and Applications to the Deep Ritz Method
- On residual minimization for PDEs: failure of PINN, modified equation, and implicit bias
- Neural base-N encoding for solving massive nonlinear systems
- Error Analysis for Solving Elliptic Interface Problems with Randomized Neural Networks
- Minimax rates for learning spectral Barron functions by deep ReLU neural networks
- Convergence analysis of controlled particle systems arising in deep learning: from finite to infinite sample size
- Fully Deterministic Shallow Neural Networks with Localized Activations. Part I: Construction and Optimal Sobolev Approximation
- Machine learning from a continuous viewpoint. I
This page was built for publication: A priori estimates of the population risk for two-layer neural networks
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2282333)
