On Lipschitz Bounds of General Convolutional Neural Networks

DOI10.1109/TIT.2019.2961812MaRDI QIDQ5123816zbMATH OpenOpenAlexFDO

Authors Dongmian Zou, Radu Balan, Maneesh K. Singh

Publication date 29 September 2020

Published in IEEE Transactions on Information Theory (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1808.01415

Mathematics Subject Classification ID

Signal theory (characterization, reconstruction, filtering, etc.) (94A12)

Abstract: Many convolutional neural networks (CNNs) have a feed-forward structure. In this paper, a linear program that estimates the Lipschitz bound of such CNNs is proposed. Several CNNs, including the scattering networks, the AlexNet and the GoogleNet, are studied numerically and compared to the theoretical bounds. Next, concentration inequalities of the output distribution to a stationary random input signal expressed in terms of the Lipschitz bound are established. The Lipschitz bound is further used to establish a nonlinear discriminant analysis designed to measure the separation between features of different classes.

Recommendations

• Lipschitz properties for deep convolutional networks
• Approximation bounds for convolutional neural networks in operator learning
• Approximation bounds for norm constrained neural networks with applications to regression and GANs
• Probabilistic Lipschitz analysis of neural networks
• A general non-Lipschitz infimal convolution regularized model: Lower bound theory and algorithm
• Łojasiewicz inequality and exponential convergence of the full-range model of CNNs
• Approximation Analysis of Convolutional Neural Networks
• Deep Neural Networks With Trainable Activations and Controlled Lipschitz Constant
• Regularisation of neural networks by enforcing Lipschitz continuity

Cited in

(11)

• Regularisation of neural networks by enforcing Lipschitz continuity
• Fast generalization error bound of deep learning without scale invariance of activation functions
• A lower bound on the competitivity of memoryless algorithms for a generalization of the CNN problem
• Lipschitz Certificates for Layered Network Structures Driven by Averaged Activation Operators
• Stable parameterization of continuous and piecewise-linear functions
• On the sensitivity of pose estimation neural networks: rotation parameterizations, Lipschitz constants, and provable bounds
• Stability of the scattering transform for deformations with minimal regularity
• Connections between numerical algorithms for PDEs and neural networks
• Generalized moduli of continuity under irregular or random deformations via multiscale analysis
• Group-invariant max filtering
• CLIP: cheap Lipschitz training of neural networks

This page was built for publication: On Lipschitz Bounds of General Convolutional Neural Networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5123816)