Stability of Deep Neural Networks via discrete rough paths

DOI10.1137/22M1472358MaRDI QIDQ6388636arXivFDO

Authors Christian Bayer, Peter Friz, Nikolas Tapia

Publication date 19 January 2022

Published in arXiv

Full work available at URL https://doi.org/10.1137/22M1472358

Mathematics Subject Classification ID

Stability theory for difference equations (39A30) Applications of rough analysis (60L90) Artificial neural networks and deep learning (68T07)

arXiv classification

cs.LGcs.NAmath.CAmath.NA

Abstract: Using rough path techniques, we provide a priori estimates for the output of Deep Residual Neural Networks in terms of both the input data and the (trained) network weights. As trained network weights are typically very rough when seen as functions of the layer, we propose to derive stability bounds in terms of the total $p$-variation of trained weights for any $pin[1,3]$. Unlike the $C^1$-theory underlying the neural ODE literature, our estimates remain bounded even in the limiting case of weights behaving like Brownian motions, as suggested in [arXiv:2105.12245]. Mathematically, we interpret residual neural network as solutions to (rough) difference equations, and analyse them based on recent results of discrete time signatures and rough path theory.

This page was built for publication: Stability of Deep Neural Networks via discrete rough paths

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6388636)