Machine learning from a continuous viewpoint. I
From MaRDI portal
Publication:829085
DOI10.1007/S11425-020-1773-8zbMATH Open1472.68136arXiv1912.12777OpenAlexW3101985406MaRDI QIDQ829085FDOQ829085
Authors: N. E. Zubov 
Publication date: 5 May 2021
Published in: Science China. Mathematics (Search for Journal in Brave)
Abstract: We present a continuous formulation of machine learning, as a problem in the calculus of variations and differential-integral equations, in the spirit of classical numerical analysis. We demonstrate that conventional machine learning models and algorithms, such as the random feature model, the two-layer neural network model and the residual neural network model, can all be recovered (in a scaled form) as particular discretizations of different continuous formulations. We also present examples of new models, such as the flow-based random feature model, and new algorithms, such as the smoothed particle method and spectral method, that arise naturally from this continuous formulation. We discuss how the issues of generalization error and implicit regularization can be studied under this framework.
Full work available at URL: https://arxiv.org/abs/1912.12777
Recommendations
- The Continuous Formulation of Shallow Neural Networks as Wasserstein-Type Gradient Flows
- The Modern Mathematics of Deep Learning
- Machine learning: deepest learning as statistical data assimilation problems
- Selection dynamics for deep neural networks
- Deep learning as optimal control problems: models and numerical methods
Learning and adaptive systems in artificial intelligence (68T05) Variational problems in a geometric measure-theoretic setting (49Q20) Numerical methods of relaxation type (49M20)
Cites Work
- Nonlinear total variation based noise removal algorithms
- DGM: a deep learning algorithm for solving partial differential equations
- Universal approximation bounds for superpositions of a sigmoidal function
- Gradient flows in metric spaces and in the space of probability measures
- Optimal Transport
- Understanding machine learning. From theory to algorithms
- The finite element methods for elliptic problems.
- Optimal approximations by piecewise smooth functions and associated variational problems
- Title not available (Why is that?)
- The Variational Formulation of the Fokker--Planck Equation
- 10.1162/153244303321897690
- Title not available (Why is that?)
- Title not available (Why is that?)
- Ridgelets: a key to higher-dimensional intermittency?
- Title not available (Why is that?)
- Approximation by superpositions of a sigmoidal function
- Harmonic analysis of neural networks
- Theory and Applications of Smoothed Particle Hydrodynamics
- \{Euclidean, metric, and Wasserstein\} gradient flows: an overview
- Solving high-dimensional partial differential equations using deep learning
- Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
- The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems
- Solving for high-dimensional committor functions using artificial neural networks
- A mean-field optimal control formulation of deep learning
- The theory of optimal processes. I. The maximum principle
- Solving the quantum many-body problem with artificial neural networks
- Solving many-electron Schrödinger equation using deep neural networks
- Stable architectures for deep neural networks
- A comparative analysis of optimization and generalization properties of two-layer neural network and random feature models under gradient descent dynamics
- A priori estimates of the population risk for two-layer neural networks
- Mean field analysis of neural networks: a central limit theorem
- Neural network with unbounded activation functions is universal approximator
- A proposal on machine learning via dynamical systems
- Maximum principle based algorithms for deep learning
- A mean field view of the landscape of two-layer neural networks
- Reconciling modern machine-learning practice and the classical bias-variance trade-off
- Machine Learning: Mathematical Theory and Scientific Applications
- Breaking the curse of dimensionality with convex neural networks
Cited In (29)
- Least action principles and well-posed learning problems
- A multi-scale DNN algorithm for nonlinear elliptic equations with multiple scales
- Representation formulas and pointwise properties for Barron functions
- Machine learning and computational mathematics
- The Random Feature Model for Input-Output Maps between Banach Spaces
- Energetic variational neural network discretizations of gradient flows
- Operator learning using random features: a tool for scientific computing
- Title not available (Why is that?)
- A rigorous framework for the mean field limit of multilayer neural networks
- Subspace decomposition based DNN algorithm for elliptic type multi-scale PDEs
- Revealing hidden dynamics from time-series data by ODENet
- The Continuous Formulation of Shallow Neural Networks as Wasserstein-Type Gradient Flows
- A Riemannian mean field formulation for two-layer neural networks with batch normalization
- Stochastic asymptotical regularization for linear inverse problems
- Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks
- Learning domain-independent Green's function for elliptic partial differential equations
- Machine learning: deepest learning as statistical data assimilation problems
- Algorithms for solving high dimensional PDEs: from nonlinear Monte Carlo to machine learning
- High order spatial discretization for variational time implicit schemes: Wasserstein gradient flows and reaction-diffusion systems
- Variational problems in machine learning and their solution with finite elements
- On the Exact Computation of Linear Frequency Principle Dynamics and Its Generalization
- Optimal sampling of parametric families: implications for machine learning
- Efficient and stable SAV-based methods for gradient flows arising from deep learning
- Title not available (Why is that?)
- Generalization error of GAN from the discriminator's perspective
- SPINN: sparse, physics-based, and partially interpretable neural networks for PDEs
- A blob method for inhomogeneous diffusion with applications to multi-agent control and sampling
- PottsMGNet: a mathematical explanation of encoder-decoder based neural networks
- A mathematical perspective of machine learning
Uses Software
This page was built for publication: Machine learning from a continuous viewpoint. I
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q829085)
