First- and second-order methods for online convolutional dictionary learning

DOI10.1137/17M1145689MaRDI QIDQ4689772zbMATH OpenWikidataFDO

Authors Jialin Liu, Cristina Garcia-Cardona, Brendt Wohlberg, Wotao Yin

Publication date 17 October 2018

Published in SIAM Journal on Imaging Sciences (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1709.00106

zbMATH Keywords

convolutional dictionary learningconvolutional sparse codingonline dictionary learningrecursive least squaresstochastic gradient descent

Mathematics Subject Classification ID

Computing methodologies for image processing (68U10) Nonconvex programming, global optimization (90C26) Image processing (compression, reconstruction, etc.) in information and communication theory (94A08) Signal theory (characterization, reconstruction, filtering, etc.) (94A12)

Abstract: Convolutional sparse representations are a form of sparse representation with a structured, translation invariant dictionary. Most convolutional dictionary learning algorithms to date operate in batch mode, requiring simultaneous access to all training images during the learning process, which results in very high memory usage and severely limits the training data that can be used. Very recently, however, a number of authors have considered the design of online convolutional dictionary learning algorithms that offer far better scaling of memory and computational cost with training set size than batch methods. This paper extends our prior work, improving a number of aspects of our previous algorithm; proposing an entirely new one, with better performance, and that supports the inclusion of a spatial mask for learning from incomplete data; and providing a rigorous theoretical analysis of these methods.

Recommendations

• Online learning for matrix factorization and sparse coding
• Efficient dictionary learning with sparseness-enforcing projections
• Toward fast transform learning
• Proximal alternating method for dictionary learning
• Dictionary Learning Algorithms for Sparse Representation

Cites work

• rm K-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation
• A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
• A fast patch-dictionary method for whole image recovery
• Atomic Decomposition by Basis Pursuit
• Convergence rate analysis of several splitting schemes
• Convolutional neural networks analyzed via convolutional sparse coding
• Efficient Algorithms for Convolutional Sparse Representations
• Exponential convergence of recursive last squares with exponential forgetting factor
• scientific article; zbMATH DE number 1818892 (Why is no real title available?)
• scientific article; zbMATH DE number 3613366 (Why is no real title available?)
• scientific article; zbMATH DE number 1465030 (Why is no real title available?)
• scientific article; zbMATH DE number 1569102 (Why is no real title available?)
• Image Super-Resolution Via Sparse Representation
• Least angle regression. (With discussion)
• Nonlinear total variation based noise removal algorithms
• Online Dictionary Learning for Kernel LMS
• Online learning for matrix factorization and sparse coding
• Proximal methods for hierarchical sparse coding
• Quasi-Martingales
• Recovery of Exact Sparse Representations in the Presence of Bounded Noise
• Recursive Least Squares Dictionary Learning Algorithm
• Sparse and Redundant Modeling of Image Content Using an Image-Signature-Dictionary
• Sparse modeling for image and vision processing
• Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic Programming
• The Theory of Max-Min, with Applications
• Unconstrained Optimization of Real Functions in Complex Variables
• Working Locally Thinking Globally: Theoretical Guarantees for Convolutional Sparse Coding

Cited in

(5)

• A dictionary learning algorithm for compression and reconstruction of streaming data in preset order
• A noise-robust online convolutional coding model and its applications to Poisson denoising and image fusion
• Correlation Based Online Dictionary Learning Algorithm
• Solving inverse problems using data-driven models
• Spatially adaptive oscillation total generalized variation for image restoration with structured textures

Describes a project that uses

Uses Software

• SPORCO
• Complex Optimization Toolbox
• PDCO

This page was built for publication: First- and second-order methods for online convolutional dictionary learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4689772)