Working Locally Thinking Globally: Theoretical Guarantees for Convolutional Sparse Coding

DOI10.1109/TSP.2017.2733447MaRDI QIDQ4621924zbMATH OpenOpenAlexFDO

Authors Vardan Papyan, Jeremias Sulam, Michael Elad

Publication date 8 February 2019

Published in IEEE Transactions on Signal Processing (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1707.06066

Mathematics Subject Classification ID

Signal theory (characterization, reconstruction, filtering, etc.) (94A12) Source coding (94A29)

Abstract: The celebrated sparse representation model has led to remarkable results in various signal processing tasks in the last decade. However, despite its initial purpose of serving as a global prior for entire signals, it has been commonly used for modeling low dimensional patches due to the computational constraints it entails when deployed with learned dictionaries. A way around this problem has been recently proposed, adopting a convolutional sparse representation model. This approach assumes that the global dictionary is a concatenation of banded Circulant matrices. While several works have presented algorithmic solutions to the global pursuit problem under this new model, very few truly-effective guarantees are known for the success of such methods. In this work, we address the theoretical aspects of the convolutional sparse model providing the first meaningful answers to questions of uniqueness of solutions and success of pursuit algorithms, both greedy and convex relaxations, in ideal and noisy regimes. To this end, we generalize mathematical quantities, such as the $el{l}_0$ norm, mutual coherence, Spark and RIP to their counterparts in the convolutional setting, intrinsically capturing local measures of the global model. On the algorithmic side, we demonstrate how to solve the global pursuit problem by using simple local processing, thus offering a first of its kind bridge between global modeling of signals and their patch-based local treatment.

Recommendations

• Convolutional neural networks analyzed via convolutional sparse coding
• A Greedy Approach to $\ell_{0,\infty}$-Based Convolutional Sparse Coding
• Variations on the Convolutional Sparse Coding Model
• Consistent Sparse Deep Learning: Theory and Computation
• On the stability of sparse convolutions
• Efficient Algorithms for Convolutional Sparse Representations
• Sparse coding for layered neural networks

Cited in

(17)

• Adversarial noise attacks of deep learning architectures: stability analysis via sparse-modeled signals
• Convergence of slice-based block coordinate descent algorithm for convolutional sparse coding
• Inference, learning and attention mechanisms that exploit and preserve sparsity in CNNs
• On locally invertible rate-1/n convolutional encoders
• Convolutional neural networks analyzed via convolutional sparse coding
• First- and second-order methods for online convolutional dictionary learning
• Maximal Function Pooling with Applications
• Multi-layer sparse coding: the holistic way
• Stability Analysis of $\ell _{0,\infty }$-Norm Based Convolutional Sparse Coding Using Stripe Coherence
• Solving inverse problems using data-driven models
• A Greedy Approach to $\ell_{0,\infty}$-Based Convolutional Sparse Coding
• Learning the sparse prior: modern approaches
• An improved theoretical analysis of convolutional sparse coding using stripe coherence
• Adaptive forward-backward splitting for multilayer convolutional dictionary learning
• Data-driven methods for quantitative imaging
• Universal architectures for the learning of polyhedral norms and convex regularizers
• Theoretical guarantees for graph sparse coding

This page was built for publication: Working Locally Thinking Globally: Theoretical Guarantees for Convolutional Sparse Coding

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4621924)