Smooth sparse coding via marginal regression for learning sparse representations

DOI10.1016/J.ARTINT.2016.04.009zbMATH Open1385.68031arXiv1210.1121OpenAlexW2614850865MaRDI QIDQ309913FDOQ309913

Authors: Krishnakumar Balasubramanian, Kai Yu, Guy Lebanon 

Publication date: 7 September 2016

Published in: Artificial Intelligence (Search for Journal in Brave)

Abstract: We propose and analyze a novel framework for learning sparse representations, based on two statistical techniques: kernel smoothing and marginal regression. The proposed approach provides a flexible framework for incorporating feature similarity or temporal information present in data sets, via non-parametric kernel smoothing. We provide generalization bounds for dictionary learning using smooth sparse coding and show how the sample complexity depends on the L1 norm of kernel function used. Furthermore, we propose using marginal regression for obtaining sparse codes, which significantly improves the speed and allows one to scale to large dictionary sizes easily. We demonstrate the advantages of the proposed approach, both in terms of accuracy and speed by extensive experimentation on several real data sets. In addition, we demonstrate how the proposed approach could be used for improving semi-supervised sparse coding.

Full work available at URL: https://arxiv.org/abs/1210.1121

Recommendations

• Online learning for matrix factorization and sparse coding
• A max-margin dictionary learning algorithm for sparse representation
• Efficient dictionary learning with sparseness-enforcing projections
• Combining Reconstruction and Discrimination with Class-Specific Sparse Coding
• Proximal alternating method for dictionary learning

zbMATH Keywords

dictionary learningsparse codingvision

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Learning and adaptive systems in artificial intelligence (68T05)

Cites Work

• Title not available (Why is that?)
• Sure Independence Screening for Ultrahigh Dimensional Feature Space
• Sparsity and Smoothness Via the Fused Lasso
• Local Regression and Likelihood
• Local Rademacher complexities
• Greed is Good: Algorithmic Results for Sparse Approximation
• A comparison of the Lasso and marginal regression
• Convergence analysis of perturbed feasible descent methods
• Combinatorial methods in density estimation
• Title not available (Why is that?)
• Smoothing \(\ell_1\)-penalized estimators for high-dimensional time-course data
• On the Goldstein-Levitin-Polyak gradient projection method
• The sample complexity of dictionary learning
• Learning with Structured Sparsity

Cited In (5)

• Hierarchical regularization networks for sparsification based learning on noisy datasets
• Provably accurate double-sparse coding
• Feature selection and multi-kernel learning for sparse representation on a manifold
• Robust sparse coding via self-paced learning for data representation
• Learning sparsely used overcomplete dictionaries via alternating minimization

Uses Software

• KernSmooth
• LIBSVM

This page was built for publication: Smooth sparse coding via marginal regression for learning sparse representations

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q309913)