Deep relaxation: partial differential equations for optimizing deep neural networks
From MaRDI portal
Publication:2319762
DOI10.1007/s40687-018-0148-yzbMath1427.82032arXiv1704.04932OpenAlexW2963480765MaRDI QIDQ2319762
Guillaume Carlier, Pratik Chaudhari, Adam M. Oberman, Stefano Soatto, Stanley J. Osher
Publication date: 20 August 2019
Published in: Research in the Mathematical Sciences (Search for Journal in Brave)
Full work available at URL: https://arxiv.org/abs/1704.04932
optimal controlneural networkspartial differential equationsstochastic gradient descentdeep learning
Numerical optimization and variational techniques (65K10) KdV equations (Korteweg-de Vries equations) (35Q53) Neural networks for/in biological studies, artificial life and related topics (92B20) Optimal stochastic control (93E20) Neural nets applied to problems in time-dependent statistical mechanics (82C32) Viscosity solutions to PDEs (35D40)
Related Items
Derivative-Free Bayesian Inversion Using Multiscale Dynamics, Solving high-dimensional Hamilton-Jacobi-Bellman PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path space, On large batch training and sharp minima: a Fokker-Planck perspective, Laplacian smoothing gradient descent, Distributed kernel gradient descent algorithm for minimum error entropy principle, A regularization interpretation of the proximal point method for weakly convex functions, Deep limits of residual neural networks, CPINNs: a coupled physics-informed neural networks for the closed-loop geothermal system, Run-and-inspect method for nonconvex optimization and global optimality bounds for R-local minimizers, Singular perturbations in stochastic optimal control with unbounded data, Markov chain stochastic DCA and applications in deep learning with PDEs regularization, Unnamed Item, A Continuous-Time Analysis of Distributed Stochastic Gradient, Accelerated Optimization in the PDE Framework Formulations for the Active Contour Case, Forward stability of ResNet and its variants, Deep neural networks motivated by partial differential equations, A mean field games approach to cluster analysis, On Bayesian posterior mean estimators in imaging sciences and Hamilton-Jacobi partial differential equations, A Multiscale Neural Network Based on Hierarchical Matrices, BinaryRelax: A Relaxation Approach for Training Deep Neural Networks with Quantized Weights, Jarzynski's equality, fluctuation theorems, and variance reduction: mathematical analysis and numerical algorithms, A multiscale neural network based on hierarchical nested bases, Stochastic Control Liaisons: Richard Sinkhorn Meets Gaspard Monge on a Schrödinger Bridge, Neural ODEs as the deep limit of ResNets with constant weights, Quantized convolutional neural networks through the lens of partial differential equations, Unnamed Item, Learning and meta-learning of stochastic advection–diffusion–reaction systems from sparse measurements, Entropic gradient descent algorithms and wide flat minima*, Biological population management based on a Hamilton–Jacobi–Bellman equation with boundary blow up
Uses Software
Cites Work
- Minimizing finite sums with the stochastic average gradient
- Semiconcave functions, Hamilton-Jacobi equations, and optimal control
- Smoothing methods for nonsmooth, nonconvex minimization
- The Fokker-Planck equation. Methods of solution and applications
- Contractions in the 2-Wasserstein length space and thermalization of granular media
- Mean field games
- Optimal transport for applied mathematicians. Calculus of variations, PDEs, and modeling
- Replica symmetry breaking condition exposed by random matrix calculation of landscape complexity
- Large population stochastic dynamic games: closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle
- Controlled Markov processes and viscosity solutions
- Local entropy as a measure for sampling solutions in constraint satisfaction problems
- Monotone Operators and the Proximal Point Algorithm
- The Variational Formulation of the Fokker--Planck Equation
- Optimization Methods for Large-Scale Machine Learning
- Stochastic Processes and Applications
- Learning representations by back-propagating errors
- Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic Programming
- Multiscale Methods
- Convergent Difference Schemes for Degenerate Elliptic and Parabolic Equations: Hamilton--Jacobi Equations and Free Boundary Problems
- Proximité et dualité dans un espace hilbertien
- A Stochastic Approximation Method
- Entropy-SGD: biasing gradient descent into wide valleys
- Inequalities: theory of majorization and its applications
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
