Neural ODE Control for Classification, Approximation, and Transport
From MaRDI portal
Abstract: We analyze Neural Ordinary Differential Equations (NODEs) from a control theoretical perspective to address some of the main properties and paradigms of Deep Learning (DL), in particular, data classification and universal approximation. These objectives are tackled and achieved from the perspective of the simultaneous control of systems of NODEs. For instance, in the context of classification, each item to be classified corresponds to a different initial datum for the control problem of the NODE, to be classified, all of them by the same common control, to the location (a subdomain of the euclidean space) associated to each label. Our proofs are genuinely nonlinear and constructive, allowing us to estimate the complexity of the control strategies we develop. The nonlinear nature of the activation functions governing the dynamics of NODEs under consideration plays a key role in our proofs, since it allows deforming half of the phase space while the other half remains invariant, a property that classical models in mechanics do not fulfill. This very property allows to build elementary controls inducing specific dynamics and transformations whose concatenation, along with properly chosen hyperplanes, allows achieving our goals in finitely many steps. The nonlinearity of the dynamics is assumed to be Lipschitz. Therefore, our results apply also in the particular case of the ReLU activation function. We also present the counterparts in the context of the control of neural transport equations, establishing a link between optimal transport and deep neural networks.
Recommendations
- Deep neural networks, generic universal interpolation, and controlled ODEs
- Deep learning approximation of diffeomorphisms via linear-control systems
- Interpolation and approximation via momentum ResNets and neural ODEs
- Deep learning as optimal control problems: models and numerical methods
- Control on the manifolds of mappings with a view to the deep learning
Cites work
- 10.1162/153244302760200704
- A Bang-Bang Theorem with Bounds on the Number of Switchings
- A computational fluid mechanics solution to the Monge-Kantorovich mass transfer problem
- A proposal on machine learning via dynamical systems
- An algorithm for total variation minimization and applications
- Approximate and exact controllability of the continuity equation with a localized vector field
- Approximation by superpositions of a sigmoidal function
- Complete controllability of continuous-time recurrent neural networks
- Computational optimal transport. With applications to data sciences
- Control on the manifolds of mappings with a view to the deep learning
- Control to flocking of the kinetic Cucker-Smale model
- Deep learning via dynamical systems: an approximation perspective
- Deep neural networks, generic universal interpolation, and controlled ODEs
- Dimensions and Measures of Quasi Self-Similar Sets
- Error bounds for approximation with neural networks
- Error bounds for approximations with deep ReLU neural networks in \(W^{s , p}\) norms
- Exact Controllability, Stabilization and Perturbations for Distributed Systems
- Extensions of Lipschitz mappings into a Hilbert space
- Fractals in probability and analysis
- From averaged to simultaneous controllability
- How violent are fast controls?
- scientific article; zbMATH DE number 3822577 (Why is no real title available?)
- scientific article; zbMATH DE number 1994007 (Why is no real title available?)
- scientific article; zbMATH DE number 1909499 (Why is no real title available?)
- scientific article; zbMATH DE number 1405266 (Why is no real title available?)
- Image recovery via total variation minimization and related problems
- Interpolation and approximation via momentum ResNets and neural ODEs
- Learning theory: stability is sufficient for generalization and necessary and sufficient for consistency of empirical risk minimization
- Matrix Optimal Mass Transport: A Quantum Mechanical Approach
- Minimal time for the continuity equation controlled by a localized perturbation of the velocity vector field
- Multilayer feedforward networks are universal approximators
- Nonlinear approximation and (deep) ReLU networks
- On Hausdorff Dimension of Recurrent Net Fractals
- Optimal observability of the multi-dimensional wave and Schrödinger equations in quantum ergodic domains
- Optimal shape and location of sensors for parabolic equations with random initial data
- Optimal Transport
- Polytopal approximation bounding the number of \(k\)-faces
- Simultaneous Exact Controllability and Some Applications
- Sparsity in long-time control of neural ODEs
- Stable architectures for deep neural networks
- The geometry of adversarial training in binary classification
- Turnpike in Lipschitz-nonlinear optimal control
- Turnpike in optimal control of PDEs, ResNets, and beyond
- Wavelet methods for pointwise regularity and local oscillations of functions
- Weak Existence for the Semigeostrophic Equations Formulated as a Coupled Monge--Ampère/Transport Problem
Cited in
(31)- A measure theoretical approach to the mean-field maximum principle for training NeurODEs
- Interpolation and approximation via momentum ResNets and neural ODEs
- Deep neural networks, generic universal interpolation, and controlled ODEs
- Deep learning approximation of diffeomorphisms via linear-control systems
- Connections between Operator-Splitting Methods and Deep Neural Networks with Applications in Image Segmentation
- Dynamical Systems–Based Neural Networks
- Control of neural transport for normalising flows
- PottsMGNet: a mathematical explanation of encoder-decoder based neural networks
- An optimal control framework for adaptive neural ODEs
- Double-well net for image segmentation
- On dissipativity of cross-entropy loss in training ResNets. A turnpike towards architecture search
- Universal approximation of dynamical systems by semiautonomous neural ODEs and applications
- Random sampling-based gradient descent method for optimal control problems with variance reduction
- Optimal transport of measures via autonomous vector fields
- Constructive approximate transport maps with normalizing flows
- Transport maps as flows of control-affine systems
- Clustering in pure-attention hardmax transformers and its role in sentiment analysis
- Local manifold approximation of dynamical system based on neural ordinary differential equation
- Cluster-based classification with neural ODEs via control
- A mathematical perspective on transformers
- Mini-workshop: Data-driven modeling, analysis, and control of dynamical systems. Abstracts from the mini-workshop held December 8--13, 2024
- Representation and regression problems in neural networks: relaxation, generalization, and numerics
- Combining physics-based and data-driven models: advancing the frontiers of research with scientific machine learning
- Normalizing flows as approximations of optimal transport maps via linear-control neural ODEs
- Interpolation, approximation, and controllability of deep neural networks
- Minimax problems for ensembles of control-affine systems
- Controllability on landmark manifolds for shapes and neural ODEs
- Optimal control of neural differential equations: the turnpike property
- Controllability of continuous networks and a kernel-based learning approximation
- A mathematical explanation of transformers
- Deep Truncated FBSDE Method: A Robust Solver for High-Dimensional Nonlinear PDEs and Fully Coupled FBSDEs
This page was built for publication: Neural ODE Control for Classification, Approximation, and Transport
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6115450)
