Lipschitz properties for deep convolutional networks

DOI10.1090/CONM/706/14205zbMATH Open1402.62224arXiv1701.05217OpenAlexW2581711117MaRDI QIDQ4686249FDOQ4686249

Maneesh K. Singh, Dongmian Zou, Radu Balan

Publication date: 9 October 2018

Published in: Contemporary Mathematics (Search for Journal in Brave)

Abstract: In this paper we discuss the stability properties of convolutional neural networks. Convolutional neural networks are widely used in machine learning. In classification they are mainly used as feature extractors. Ideally, we expect similar features when the inputs are from the same class. That is, we hope to see a small change in the feature vector with respect to a deformation on the input signal. This can be established mathematically, and the key step is to derive the Lipschitz properties. Further, we establish that the stability results can be extended for more general networks. We give a formula for computing the Lipschitz bound, and compare it with other methods to show it is closer to the optimal value.

Full work available at URL: https://arxiv.org/abs/1701.05217

Recommendations

• CLIP: cheap Lipschitz training of neural networks
• Group invariance, stability to deformations, and complexity of deep convolutional representations
• Universality of deep convolutional neural networks
• Approximation properties of deep ReLU CNNs
• Stability of Deep Neural Networks via Discrete Rough Paths

zbMATH Keywords

classificationmachine learningconvolutional neural networksLipschitz properties

Mathematics Subject Classification ID

Classification and discrimination; cluster analysis (statistical aspects) (62H30) Learning and adaptive systems in artificial intelligence (68T05) Neural nets and related approaches to inference from stochastic processes (62M45)

Cites Work

• Title not available (Why is that?)
• Group invariant scattering
• A Mathematical Motivation for Complex-Valued Convolutional Networks
• A Mathematical Theory of Deep Convolutional Neural Networks for Feature Extraction

Cited In (13)

• A Note on the Regularity of Images Generated by Convolutional Neural Networks
• Stability of the scattering transform for deformations with minimal regularity
• Rendition: Reclaiming What a Black Box Takes Away
• A noise-based stabilizer for convolutional neural networks
• On the convergence of formally diverging neural net-based classifiers
• Stability for the training of deep neural networks and other classifiers
• Regularisation of neural networks by enforcing Lipschitz continuity
• Asymptotic analysis of higher-order scattering transform of Gaussian processes
• Stability of iterated dyadic filter banks
• Stochastic Markov gradient descent and training low-bit neural networks
• Lipschitz Certificates for Layered Network Structures Driven by Averaged Activation Operators
• On Lipschitz Bounds of General Convolutional Neural Networks
• Gabor neural networks with proven approximation properties

This page was built for publication: Lipschitz properties for deep convolutional networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4686249)