Stability of Deep Neural Networks via Discrete Rough Paths

DOI10.1137/22M1472358MaRDI QIDQ5885832zbMATH OpenFDO

Authors Christian Bayer, Peter Friz, Nikolas Tapia

Publication date 30 March 2023

Published in SIAM Journal on Mathematics of Data Science (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2201.07566

zbMATH Keywords

\(p\)-variationresidual neural networksrough pathsstability

Mathematics Subject Classification ID

Stability theory for difference equations (39A30) Applications of rough analysis (60L90) Artificial neural networks and deep learning (68T07)

Recommendations

• Forward stability of ResNet and its variants
• Continuous limits of residual neural networks in case of large input data
• Stochastic training of residual networks in deep learning
• Stable architectures for deep neural networks
• Selection dynamics for deep neural networks

Cites work

• A priori estimates for rough PDEs with application to rough conservation laws
• A proposal on machine learning via dynamical systems
• Controlling rough paths
• Curvilinear integrals along enriched paths
• Differential Equations Driven by Rough Paths: An Approach via Discrete Approximation
• Differential equations driven by rough signals
• Geometric versus non-geometric rough paths
• Integrability and tail estimates for Gaussian rough differential equations
• Multidimensional stochastic processes as rough paths. Theory and applications.
• Stable architectures for deep neural networks
• Strong and weak approximation of semilinear stochastic evolution equations
• Time-warping invariants of multidimensional time series

Cited in

(16)

• Forward stability of ResNet and its variants
• Stable architectures for deep neural networks
• Lipschitz properties for deep convolutional networks
• On the S-instability and degeneracy of discrete deep learning models
• Nonlinear weighted directed acyclic graph and a priori estimates for neural networks
• Stochastic training of residual networks in deep learning
• Noise sensitivity and stability of deep neural networks for binary classification
• Stability of Deep Neural Networks via discrete rough paths
• Signature methods in machine learning
• A gradient flow on control space with rough initial condition
• Iterated ergodic theorems and Erdös-Rényi law of large numbers
• Almost sure diffusion approximation in averaging: direct proofs with rough paths flavors
• Directions in rough analysis. Abstracts from the workshop held November 3--8, 2024
• Scaling ResNets in the large-depth regime
• Numerical attractors via discrete rough paths
• CLIP: cheap Lipschitz training of neural networks

Describes a project that uses

Uses Software

• NeuralCDE
• torchcde
• Diffrax

This page was built for publication: Stability of Deep Neural Networks via Discrete Rough Paths

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5885832)