Lipschitz properties for deep convolutional networks

DOI10.1090/CONM/706/14205MaRDI QIDQ4686249zbMATH OpenOpenAlexFDO

Authors Radu Balan, Dongmian Zou, Maneesh K. Singh

Publication date 9 October 2018

Published in Contemporary Mathematics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1701.05217

zbMATH Keywords

classificationmachine learningconvolutional neural networksLipschitz properties

Mathematics Subject Classification ID

Classification and discrimination; cluster analysis (statistical aspects) (62H30) Learning and adaptive systems in artificial intelligence (68T05) Neural nets and related approaches to inference from stochastic processes (62M45)

Abstract: In this paper we discuss the stability properties of convolutional neural networks. Convolutional neural networks are widely used in machine learning. In classification they are mainly used as feature extractors. Ideally, we expect similar features when the inputs are from the same class. That is, we hope to see a small change in the feature vector with respect to a deformation on the input signal. This can be established mathematically, and the key step is to derive the Lipschitz properties. Further, we establish that the stability results can be extended for more general networks. We give a formula for computing the Lipschitz bound, and compare it with other methods to show it is closer to the optimal value.

Recommendations

• CLIP: cheap Lipschitz training of neural networks
• Group invariance, stability to deformations, and complexity of deep convolutional representations
• Universality of deep convolutional neural networks
• Approximation properties of deep ReLU CNNs
• Stability of Deep Neural Networks via Discrete Rough Paths

Cites work

• A Mathematical Theory of Deep Convolutional Neural Networks for Feature Extraction
• A mathematical motivation for complex-valued convolutional networks
• Deep learning
• Group invariant scattering

Cited in

(18)

• Lipschitz Certificates for Layered Network Structures Driven by Averaged Activation Operators
• Gabor neural networks with proven approximation properties
• Probabilistic Lipschitz analysis of neural networks
• Stability for the training of deep neural networks and other classifiers
• Designing stable neural networks using convex analysis and ODEs
• Stability of iterated dyadic filter banks
• A noise-based stabilizer for convolutional neural networks
• CLIP: cheap Lipschitz training of neural networks
• Asymptotic analysis of higher-order scattering transform of Gaussian processes
• Regularisation of neural networks by enforcing Lipschitz continuity
• Stochastic Markov gradient descent and training low-bit neural networks
• A Note on the Regularity of Images Generated by Convolutional Neural Networks
• On Lipschitz Bounds of General Convolutional Neural Networks
• Stability of the scattering transform for deformations with minimal regularity
• Multilinear compressive sensing and an application to convolutional linear networks
• On the convergence of formally diverging neural net-based classifiers
• Group invariance, stability to deformations, and complexity of deep convolutional representations
• Rendition: reclaiming what a black box takes away

This page was built for publication: Lipschitz properties for deep convolutional networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4686249)