Nowhere coexpanding functions

DOI10.1063/5.0162148MaRDI QIDQ6430509zbMATH OpenOpenAlexFDO

Authors Andrew W. Cook, Andy Hammerlindl, W. Tucker

Publication date 22 March 2023

Full work available at URL https://doi.org/10.1063/5.0162148

Mathematics Subject Classification ID

Qualitative theory for ordinary differential equations (34Cxx) Smooth dynamical systems: general theory (37Cxx) Low-dimensional dynamical systems (37Exx)

Abstract: In this paper, we establish a sharp upper bound on the the number of fixed points a certain class of neural networks can have. The networks under study (autoencoders) can be viewed as discrete dynamical systems whose nonlinearities are given by the choice of activation functions. To this end, we introduce a new class $mathcalF$ of $C^1$ activation functions that is closed under composition, and contains e.g. the logistic sigmoid function. We use this class to show that any 1-dimensional neural network of arbitrary depth with activation functions in $mathcalF$ has at most three fixed points. Due to the simple nature of such networks, we are able to completely understand their fixed points, providing a foundation to the much needed connection between application and theory of deep neural networks.

This page was built for publication: Nowhere coexpanding functions

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6430509)