An MDL Framework for Sparse Coding and Dictionary Learning

DOI10.1109/TSP.2012.2187203MaRDI QIDQ4573755zbMATH OpenFDO

Authors Ignacio Ramírez, Guillermo Sapiro

Publication date 18 July 2018

Published in IEEE Transactions on Signal Processing (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1110.2436

Mathematics Subject Classification ID

Statistical aspects of information-theoretic topics (62B10) Sampling theory in information and communication theory (94A20)

Abstract: The power of sparse signal modeling with learned over-complete dictionaries has been demonstrated in a variety of applications and fields, from signal processing to statistical inference and machine learning. However, the statistical properties of these models, such as under-fitting or over-fitting given sets of data, are still not well characterized in the literature. As a result, the success of sparse modeling depends on hand-tuning critical parameters for each data and application. This work aims at addressing this by providing a practical and objective characterization of sparse models by means of the Minimum Description Length (MDL) principle -- a well established information-theoretic approach to model selection in statistical inference. The resulting framework derives a family of efficient sparse coding and dictionary learning algorithms which, by virtue of the MDL principle, are completely parameter free. Furthermore, such framework allows to incorporate additional prior information to existing models, such as Markovian dependencies, or to define completely new problem formulations, including in the matrix analysis area, in a natural way. These virtues will be demonstrated with parameter-free algorithms for the classic image denoising and classification problems, and for low-rank matrix recovery in video applications.

Recommendations

• Dictionary Learning for L1-Exact Sparse Coding
• Multi-Layer Convolutional Sparse Modeling: Pursuit and Dictionary Learning
• Low-rank and sparse dictionary learning
• Dictionary Learning for Sparse Approximations With the Majorization Method
• Efficient dictionary learning with sparseness-enforcing projections
• A max-margin dictionary learning algorithm for sparse representation
• Dictionary Learning Algorithms for Sparse Representation

Cited in

(7)

• Designing Structured Sparse Dictionaries for Sparse Representation Modeling
• Dictionary Learning for L1-Exact Sparse Coding
• Sparse Codes Auto-Extractor for Classification: A Joint Embedding and Dictionary Learning Framework for Representation
• Multi-Layer Convolutional Sparse Modeling: Pursuit and Dictionary Learning
• Learning and Inference in Sparse Coding Models With Langevin Dynamics
• A tree-based dictionary learning framework
• Decomposition into low-rank plus additive matrices for background/foreground separation: a review for a comparative evaluation with a large-scale dataset

This page was built for publication: An MDL Framework for Sparse Coding and Dictionary Learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4573755)