Multi-Layer Convolutional Sparse Modeling: Pursuit and Dictionary Learning
From MaRDI portal
Abstract: The recently proposed Multi-Layer Convolutional Sparse Coding (ML-CSC) model, consisting of a cascade of convolutional sparse layers, provides a new interpretation of Convolutional Neural Networks (CNNs). Under this framework, the computation of the forward pass in a CNN is equivalent to a pursuit algorithm aiming to estimate the nested sparse representation vectors -- or feature maps -- from a given input signal. Despite having served as a pivotal connection between CNNs and sparse modeling, a deeper understanding of the ML-CSC is still lacking: there are no pursuit algorithms that can serve this model exactly, nor are there conditions to guarantee a non-empty model. While one can easily obtain signals that approximately satisfy the ML-CSC constraints, it remains unclear how to simply sample from the model and, more importantly, how one can train the convolutional filters from real data. In this work, we propose a sound pursuit algorithm for the ML-CSC model by adopting a projection approach. We provide new and improved bounds on the stability of the solution of such pursuit and we analyze different practical alternatives to implement this in practice. We show that the training of the filters is essential to allow for non-trivial signals in the model, and we derive an online algorithm to learn the dictionaries from real data, effectively resulting in cascaded sparse convolutional layers. Last, but not least, we demonstrate the applicability of the ML-CSC model for several applications in an unsupervised setting, providing competitive results. Our work represents a bridge between matrix factorization, sparse dictionary learning and sparse auto-encoders, and we analyze these connections in detail.
Recommendations
- An MDL Framework for Sparse Coding and Dictionary Learning
- Convolutional Dictionary Learning: Acceleration and Convergence
- Efficient Algorithms for Convolutional Sparse Representations
- Variations on the Convolutional Sparse Coding Model
- Bag of pursuits and neural gas for improved sparse coding
- Multi-layer sparse coding: the holistic way
- Convolutional neural networks analyzed via convolutional sparse coding
- A Greedy Approach to $\ell_{0,\infty}$-Based Convolutional Sparse Coding
- Dictionary Learning for L1-Exact Sparse Coding
- scientific article; zbMATH DE number 7195210
Cited in
(14)- Adversarial noise attacks of deep learning architectures: stability analysis via sparse-modeled signals
- A convex variational model for learning convolutional image atoms from incomplete data
- Convergence of slice-based block coordinate descent algorithm for convolutional sparse coding
- Inference, learning and attention mechanisms that exploit and preserve sparsity in CNNs
- An MDL Framework for Sparse Coding and Dictionary Learning
- Multi-layer sparse coding: the holistic way
- Deep Learning--Based Dictionary Learning and Tomographic Image Reconstruction
- A Generative Variational Model for Inverse Problems in Imaging
- Convergence of a piggyback-style method for the differentiation of solutions of standard saddle-point problems
- Solving inverse problems using data-driven models
- A Greedy Approach to $\ell_{0,\infty}$-Based Convolutional Sparse Coding
- Learning the sparse prior: modern approaches
- An improved theoretical analysis of convolutional sparse coding using stripe coherence
- Adaptive forward-backward splitting for multilayer convolutional dictionary learning
This page was built for publication: Multi-Layer Convolutional Sparse Modeling: Pursuit and Dictionary Learning
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4622304)
