Machine learning from a continuous viewpoint. I
From MaRDI portal
(Redirected from Publication:829085)
Abstract: We present a continuous formulation of machine learning, as a problem in the calculus of variations and differential-integral equations, in the spirit of classical numerical analysis. We demonstrate that conventional machine learning models and algorithms, such as the random feature model, the two-layer neural network model and the residual neural network model, can all be recovered (in a scaled form) as particular discretizations of different continuous formulations. We also present examples of new models, such as the flow-based random feature model, and new algorithms, such as the smoothed particle method and spectral method, that arise naturally from this continuous formulation. We discuss how the issues of generalization error and implicit regularization can be studied under this framework.
Recommendations
- The Continuous Formulation of Shallow Neural Networks as Wasserstein-Type Gradient Flows
- The Modern Mathematics of Deep Learning
- Machine learning: deepest learning as statistical data assimilation problems
- Selection dynamics for deep neural networks
- Deep learning as optimal control problems: models and numerical methods
Cites work
- scientific article; zbMATH DE number 140601 (Why is no real title available?)
- scientific article; zbMATH DE number 3640828 (Why is no real title available?)
- scientific article; zbMATH DE number 872523 (Why is no real title available?)
- scientific article; zbMATH DE number 3250158 (Why is no real title available?)
- 10.1162/153244303321897690
- A comparative analysis of optimization and generalization properties of two-layer neural network and random feature models under gradient descent dynamics
- A mean field view of the landscape of two-layer neural networks
- A mean-field optimal control formulation of deep learning
- A priori estimates of the population risk for two-layer neural networks
- A proposal on machine learning via dynamical systems
- Approximation by superpositions of a sigmoidal function
- Breaking the curse of dimensionality with convex neural networks
- DGM: a deep learning algorithm for solving partial differential equations
- Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
- Gradient flows in metric spaces and in the space of probability measures
- Harmonic analysis of neural networks
- Machine Learning: Mathematical Theory and Scientific Applications
- Maximum principle based algorithms for deep learning
- Mean field analysis of neural networks: a central limit theorem
- Neural network with unbounded activation functions is universal approximator
- Nonlinear total variation based noise removal algorithms
- Optimal Transport
- Optimal approximations by piecewise smooth functions and associated variational problems
- Reconciling modern machine-learning practice and the classical bias-variance trade-off
- Ridgelets: a key to higher-dimensional intermittency?
- Solving for high-dimensional committor functions using artificial neural networks
- Solving high-dimensional partial differential equations using deep learning
- Solving many-electron Schrödinger equation using deep neural networks
- Solving the quantum many-body problem with artificial neural networks
- Stable architectures for deep neural networks
- The Variational Formulation of the Fokker--Planck Equation
- The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems
- The finite element methods for elliptic problems.
- The theory of optimal processes. I. The maximum principle
- Theory and Applications of Smoothed Particle Hydrodynamics
- Understanding machine learning. From theory to algorithms
- Universal approximation bounds for superpositions of a sigmoidal function
- \{Euclidean, metric, and Wasserstein\} gradient flows: an overview
Cited in
(29)- Least action principles and well-posed learning problems
- A multi-scale DNN algorithm for nonlinear elliptic equations with multiple scales
- Representation formulas and pointwise properties for Barron functions
- Machine learning and computational mathematics
- The Random Feature Model for Input-Output Maps between Banach Spaces
- scientific article; zbMATH DE number 7370564 (Why is no real title available?)
- Energetic variational neural network discretizations of gradient flows
- Operator learning using random features: a tool for scientific computing
- A rigorous framework for the mean field limit of multilayer neural networks
- Subspace decomposition based DNN algorithm for elliptic type multi-scale PDEs
- Revealing hidden dynamics from time-series data by ODENet
- The Continuous Formulation of Shallow Neural Networks as Wasserstein-Type Gradient Flows
- A Riemannian mean field formulation for two-layer neural networks with batch normalization
- Stochastic asymptotical regularization for linear inverse problems
- Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks
- Learning domain-independent Green's function for elliptic partial differential equations
- Machine learning: deepest learning as statistical data assimilation problems
- Algorithms for solving high dimensional PDEs: from nonlinear Monte Carlo to machine learning
- Variational problems in machine learning and their solution with finite elements
- High order spatial discretization for variational time implicit schemes: Wasserstein gradient flows and reaction-diffusion systems
- On the Exact Computation of Linear Frequency Principle Dynamics and Its Generalization
- Optimal sampling of parametric families: implications for machine learning
- Efficient and stable SAV-based methods for gradient flows arising from deep learning
- Generalization error of GAN from the discriminator's perspective
- scientific article; zbMATH DE number 7625201 (Why is no real title available?)
- SPINN: sparse, physics-based, and partially interpretable neural networks for PDEs
- A blob method for inhomogeneous diffusion with applications to multi-agent control and sampling
- PottsMGNet: a mathematical explanation of encoder-decoder based neural networks
- A mathematical perspective of machine learning
This page was built for publication: Machine learning from a continuous viewpoint. I
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q829085)
