Structure-preserving deep learning

DOI10.1017/S0956792521000139WikidataQ124988461 ScholiaQ124988461MaRDI QIDQ5014474FDOQ5014474

Authors: Elena Celledoni, Matthias J. Ehrhardt, Christian Etmann, Brynjulf Owren, Carola-Bibiane Schönlieb, Ferdia Sherry, Robert I. McLachlan 

Publication date: 8 December 2021

Published in: European Journal of Applied Mathematics (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2006.03364

Recommendations

• The Modern Mathematics of Deep Learning
• An introduction to the mathematics of deep learning
• Deep learning as optimal control problems: models and numerical methods
• Stable architectures for deep neural networks
• Dynamical Systems andOptimal Control Approach to Deep Learning

zbMATH Keywords

deep learningoptimal controlordinary differential equationsstructure-preserving methods

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Numerical methods in optimal control (49Mxx) Numerical methods for ordinary differential equations (65Lxx) Numerical methods for mathematical programming, optimization and variational techniques (65Kxx)

Cites Work

• Solving ordinary differential equations. II: Stiff and differential-algebraic problems.
• Measure theory. Vol. I and II
• Title not available (Why is that?)
• Title not available (Why is that?)
• Gradient flows in metric spaces and in the space of probability measures
• A Stochastic Approximation Method
• Title not available (Why is that?)
• Understanding machine learning. From theory to algorithms
• Runge-Kutta methods in optimal control and the transformed adjoint system
• Solving Ordinary Differential Equations I
• Trust Region Methods
• Title not available (Why is that?)
• Neural networks and physical systems with emergent collective computational abilities
• Geometric Numerical Integration
• Title not available (Why is that?)
• Riemannian geometry
• Continuum limit of total variation on point clouds
• Port-Hamiltonian systems theory: an introductory overview
• Geometric integration of non-autonomous linear Hamiltonian problems
• Generalized canonical transformations for time-dependent systems
• Splitting methods
• Geometric integration using discrete gradients
• A stochastic estimator of the trace of the influence matrix for laplacian smoothing splines
• Approximation by superpositions of a sigmoidal function
• Title not available (Why is that?)
• An introduction to Lie group integrators - basics, new developments and applications
• Total variation regularization for manifold-valued data
• Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion
• Neural learning by geometric integration of reduced `rigid-body' equations
• Conformal Hamiltonian systems
• A mean-field optimal control formulation of deep learning
• Second order conformal symplectic schemes for damped Hamiltonian systems
• Optimal approximation with sparsely connected deep neural networks
• Forward stability of ResNet and its variants
• Deep neural networks motivated by partial differential equations
• Stable architectures for deep neural networks
• Solving inverse problems using data-driven models
• A proposal on machine learning via dynamical systems
• Maximum principle based algorithms for deep learning
• Stochastic modified equations and dynamics of stochastic gradient algorithms. I: Mathematical foundations
• Analysis of the BFGS Method with Errors
• Flat Minima
• Deep learning as optimal control problems: models and numerical methods
• Shape analysis on Lie groups with applications in computer animation
• Stochastic Quasi-Newton Methods for Nonconvex Stochastic Optimization
• Geometry of matrix decompositions seen through optimal transport and information geometry
• Inverse problems. Tikhonov theory and algorithms
• Layer-Parallel Training of Deep Residual Neural Networks
• Conformal symplectic and relativistic optimization
• Equivalence of approximation by convolutional neural networks and fully-connected networks
• Title not available (Why is that?)
• log det A = tr log A

Cited In (21)

• Sparsity in long-time control of neural ODEs
• Structure preserving deep learning
• Deep limits of residual neural networks
• Deep learning for inverse problems. Abstracts from the workshop held March 7--13, 2021 (hybrid meeting)
• Turnpike in optimal control of PDEs, ResNets, and beyond
• Locally-symplectic neural networks for learning volume-preserving dynamics
• Identifying the group-theoretic structure of machine-learned symmetries
• VPNets: volume-preserving neural networks for learning source-free dynamics
• Designing universal causal deep learning models: The geometric (Hyper)transformer
• Image Denoising: The Deep Learning Revolution and Beyond—A Survey Paper
• Algorithms for solving high dimensional PDEs: from nonlinear Monte Carlo to machine learning
• Classification with Runge-Kutta networks and feature space augmentation
• Deep neural networks on diffeomorphism groups for optimal shape reparametrization
• The mpEDMD Algorithm for Data-Driven Computations of Measure-Preserving Dynamical Systems
• Geometric numerical integration. Abstracts from the workshop held March 28 -- April 3, 2021 (hybrid meeting)
• Inverse evolution layers: physics-informed regularizers for image segmentation
• Constrained dynamics, stochastic numerical methods and the modeling of complex systems. Abstracts from the workshop held May 26--31, 2024
• PottsMGNet: a mathematical explanation of encoder-decoder based neural networks
• Connectivity preserving network transformers
• Discrete gradients in short-range molecular dynamics simulations
• Dynamical Systems–Based Neural Networks

Uses Software

• RODAS
• Camino
• ImageNet
• Adam
• NICE
• i-RevNet
• Glow
• nflows
• nsf
• torchdiffeq
• ANODEs
• AlexNet
• ANODE
• StyleGAN

This page was built for publication: Structure-preserving deep learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5014474)