Control on the manifolds of mappings with a view to the deep learning

DOI10.1007/S10883-021-09561-2MaRDI QIDQ2676673zbMATH OpenOpenAlexFDO

Authors A. V. Sarychev, A. A. Agrachev

Publication date 28 September 2022

Published in Journal of Dynamical and Control Systems (Search for Journal in Brave)

Copyright license Creative Commons Attribution 4.0 International

Full work available at URL https://arxiv.org/abs/2008.12702

zbMATH Keywords

controllability on manifolds of mappingsdeep learningensemble controllabilityoptimal control

Mathematics Subject Classification ID

Manifolds of mappings (58D15) Artificial neural networks and deep learning (68T07) Controllability (93B05) Geometric methods (93B27)

Abstract: Deep learning of the Artificial Neural Networks (ANN) can be treated as a particular class of interpolation problems. The goal is to find a neural network whose input-output map approximates well the desired map on a finite or an infinite training set. Our idea consists of taking as an approximant the input-output map, which arises from a nonlinear continuous-time control system. In the limit such control system can be seen as a network with a continuum of layers, each one labelled by the time variable. The values of the controls at each instant of time are the parameters of the layer.

Recommendations

• Deep learning approximation of diffeomorphisms via linear-control systems
• Deep learning as optimal control problems: models and numerical methods
• Dynamical Systems andOptimal Control Approach to Deep Learning
• Deep neural networks, generic universal interpolation, and controlled ODEs
• A mean-field optimal control formulation of deep learning

Cites work

• A Pseudospectral Three-Dimensional Boundary Integral Method Applied to a Nonlinear Model Problem from Finite Elasticity
• Control in the spaces of ensembles of points
• Control theory from the geometric viewpoint.
• Deep learning: an introduction for applied mathematicians
• Deep neural networks, generic universal interpolation, and controlled ODEs
• scientific article; zbMATH DE number 2113701 (Why is no real title available?)
• Maximum principle based algorithms for deep learning
• Orthogonal polynomials of several variables
• Solid Controllability in Fluid Dynamics
• THE EXPONENTIAL REPRESENTATION OF FLOWS AND THE CHRONOLOGICAL CALCULUS

Cited in

(21)

• A mean-field optimal control formulation of deep learning
• On fixed points of continuous mappings associated with construction of artificial neural networks
• Deep neural networks, generic universal interpolation, and controlled ODEs
• Turnpike in optimal control of PDEs, ResNets, and beyond
• Deep neural networks on diffeomorphism groups for optimal shape reparametrization
• Deep learning approximation of diffeomorphisms via linear-control systems
• Neural ODE Control for Classification, Approximation, and Transport
• ``Good Lie brackets for control affine systems
• Dynamical Systems–Based Neural Networks
• Control of neural transport for normalising flows
• Universal approximation of dynamical systems by semiautonomous neural ODEs and applications
• A gradient flow on control space with rough initial condition
• Sub-Riemannian landmark matching and its interpretation as residual neural networks
• Constructive approximate transport maps with normalizing flows
• Transport maps as flows of control-affine systems
• A mathematical perspective on transformers
• Normalizing flows as approximations of optimal transport maps via linear-control neural ODEs
• Interpolation, approximation, and controllability of deep neural networks
• Minimax problems for ensembles of control-affine systems
• Controllability on landmark manifolds for shapes and neural ODEs
• Flows on measure spaces and applications in machine learning. Abstracts from the workshop held March 22--27, 2026

This page was built for publication: Control on the manifolds of mappings with a view to the deep learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2676673)