Lipschitz properties for deep convolutional networks

DOI10.1090/CONM/706/14205MaRDI QIDQ4686249zbMATH OpenOpenAlexFDO

Authors Radu Balan, Dongmian Zou, Maneesh K. Singh

Publication date 9 October 2018

Published in Contemporary Mathematics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1701.05217

zbMATH Keywords

classificationconvolutional neural networksLipschitz propertiesmachine learning

Mathematics Subject Classification ID

Classification and discrimination; cluster analysis (statistical aspects) (62H30) Neural nets and related approaches to inference from stochastic processes (62M45) Learning and adaptive systems in artificial intelligence (68T05)

Abstract: In this paper we discuss the stability properties of convolutional neural networks. Convolutional neural networks are widely used in machine learning. In classification they are mainly used as feature extractors. Ideally, we expect similar features when the inputs are from the same class. That is, we hope to see a small change in the feature vector with respect to a deformation on the input signal. This can be established mathematically, and the key step is to derive the Lipschitz properties. Further, we establish that the stability results can be extended for more general networks. We give a formula for computing the Lipschitz bound, and compare it with other methods to show it is closer to the optimal value.

Recommendations

• CLIP: cheap Lipschitz training of neural networks
• Group invariance, stability to deformations, and complexity of deep convolutional representations
• Universality of deep convolutional neural networks
• Approximation properties of deep ReLU CNNs
• Stability of Deep Neural Networks via Discrete Rough Paths

Cites work

• A mathematical motivation for complex-valued convolutional networks
• A Mathematical Theory of Deep Convolutional Neural Networks for Feature Extraction
• Deep learning
• Group invariant scattering

Cited in

(20)

• On the convergence of formally diverging neural net-based classifiers
• Gabor neural networks with proven approximation properties
• Regularisation of neural networks by enforcing Lipschitz continuity
• Stochastic Markov gradient descent and training low-bit neural networks
• Asymptotic analysis of higher-order scattering transform of Gaussian processes
• Probabilistic Lipschitz analysis of neural networks
• Stability of iterated dyadic filter banks
• Group invariance, stability to deformations, and complexity of deep convolutional representations
• Stability for the training of deep neural networks and other classifiers
• Multilinear compressive sensing and an application to convolutional linear networks
• Lipschitz Certificates for Layered Network Structures Driven by Averaged Activation Operators
• A noise-based stabilizer for convolutional neural networks
• On Lipschitz Bounds of General Convolutional Neural Networks
• Rendition: reclaiming what a black box takes away
• A Note on the Regularity of Images Generated by Convolutional Neural Networks
• Stability of the scattering transform for deformations with minimal regularity
• Designing stable neural networks using convex analysis and ODEs
• Generalized moduli of continuity under irregular or random deformations via multiscale analysis
• The problem of accuracy in systems for cyberattack resistance and the verification of neural networks on the example of botnet detecting problem
• CLIP: cheap Lipschitz training of neural networks

This page was built for publication: Lipschitz properties for deep convolutional networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4686249)