Deep learning via dynamical systems: an approximation perspective (Q6172665): Difference between revisions

An important challenge in recent trends in deep learning is to develop a framework for its effectiveness by capturing the effect of sequential function composition in deep neural networks and one important tool used in this regard is a dynamical systems approach whereby one thinks of deep neural networks as a discretization of certain ordinary differential equations. In this interesting paper, the authors study the dynamical systems approach whereby deep residual networks are idealized as continuous-time dynamical systems, from an approximation perspective. The authors prove several theorems which give sufficient conditions for universal approximation using continuous time deep residual networks which can also be understood as approximation theories in \(L_p\) using certain flow maps of dynamical systems. The authors also establish rates of approximation in terms of some time horizons. This is a new and interesting idea in approximation theory, namely that composition function approximation through flow maps contributes to building useful mathematical frameworks to investigate deep learning. The paper is well written with ample references covering both classical and newer work on deep neural networks, their connections to approximation theory, control and differential equations.