Deep learning as optimal control problems: models and numerical methods

DOI10.3934/jcd.2019009zbMath1429.68249arXiv1904.05657OpenAlexW2990121169WikidataQ126645840 ScholiaQ126645840MaRDI QIDQ2297872

Brynjulf Owren, Martin Benning, Matthias J. Ehrhardt, Elena Celledoni, Carola-Bibiane Schönlieb

Publication date: 20 February 2020

Published in: Journal of Computational Dynamics (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1904.05657

zbMATH Keywords

optimal controlRunge-Kutta methodsdeep learningHamiltonian boundary value problems

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Numerical optimization and variational techniques (65K10) Existence theories for optimal control problems involving ordinary differential equations (49J15) Numerical solution of boundary value problems involving ordinary differential equations (65L10) Numerical methods for Hamiltonian systems including symplectic integrators (65P10) Discrete approximations in optimal control (49M25)

Related Items (18)

Turnpike in optimal control of PDEs, ResNets, and beyond ⋮ Deep learning for inverse problems. Abstracts from the workshop held March 7--13, 2021 (hybrid meeting) ⋮ A framework for randomized time-splitting in linear-quadratic optimal control ⋮ Deep neural networks on diffeomorphism groups for optimal shape reparametrization ⋮ Deep learning approximation of diffeomorphisms via linear-control systems ⋮ Sparsity in long-time control of neural ODEs ⋮ Accuracy Estimates for Bilinear Optimal Control Problems Governed by Ordinary Differential Equations ⋮ Unnamed Item ⋮ Optimal control by deep learning techniques and its applications on epidemic models ⋮ Geometric methods for adjoint systems ⋮ Optimal Dirichlet boundary control by Fourier neural operators applied to nonlinear optics ⋮ A projected primal-dual gradient optimal control method for deep reinforcement learning ⋮ Variational networks: an optimal control approach to early stopping variational methods for image restoration ⋮ Preface. Special issue in honor of Reinout Quispel ⋮ Classification with Runge-Kutta networks and feature space augmentation ⋮ Structure-preserving deep learning ⋮ A measure theoretical approach to the mean-field maximum principle for training NeurODEs ⋮ Algorithms for solving high dimensional PDEs: from nonlinear Monte Carlo to machine learning

Uses Software

• DeepFool
• AlphaGo
• torchdiffeq
• ANODE
• PRMLT

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Clebsch optimal control formulation in mechanics
• Runge-Kutta methods in optimal control and the transformed adjoint system
• On the necessity of negative coefficients for operator splitting schemes of order higher than two
• Control theory from the geometric viewpoint.
• A mean-field optimal control formulation of deep learning
• Neural network with unbounded activation functions is universal approximator
• A proposal on machine learning via dynamical systems
• Nonlinear inverse scale space methods
• Deep limits of residual neural networks
• Symplectic Runge--Kutta Schemes for Adjoint Equations, Automatic Differentiation, Optimal Control, and More
• Linear Inversion of Band-Limited Reflection Seismograms
• A Technique for the Numerical Solution of Certain Integral Equations of the First Kind
• The Euler approximation in state constrained optimal control
• Stable architectures for deep neural networks
• Artificial intelligence as structural estimation: Deep Blue, Bonanza, and AlphaGo
• Machine Learning: Deepest Learning as Statistical Data Assimilation Problems
• Modern regularization methods for inverse problems
• Deep Learning: An Introduction for Applied Mathematicians
• Geometric Numerical Integration
• Variational, Geometric, and Level Set Methods in Computer Vision
• Two-Point Boundary Value Problems of Linear Hamiltonian Systems

This page was built for publication: Deep learning as optimal control problems: models and numerical methods