Adaptive multi-penalty regularization based on a generalized Lasso path

DOI10.1016/J.ACHA.2018.11.001MaRDI QIDQ2175012zbMATH OpenOpenAlexWikidataFDO

Authors Markus Grasmair, Timo Klock, Valeriya Naumova

Publication date 27 April 2020

Published in Applied and Computational Harmonic Analysis (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1710.03971

zbMATH Keywords

compressed sensingadaptive parameter choicemulti-penalty regularizationLasso pathexact support recoverynoise folding

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Ridge regression; shrinkage estimators (Lasso) (62J07) Signal theory (characterization, reconstruction, filtering, etc.) (94A12)

Abstract: For many algorithms, parameter tuning remains a challenging and critical task, which becomes tedious and infeasible in a multi-parameter setting. Multi-penalty regularization, successfully used for solving undetermined sparse regression of problems of unmixing type where signal and noise are additively mixed, is one of such examples. In this paper, we propose a novel algorithmic framework for an adaptive parameter choice in multi-penalty regularization with a focus on the correct support recovery. Building upon the theory of regularization paths and algorithms for single-penalty functionals, we extend these ideas to a multi-penalty framework by providing an efficient procedure for the construction of regions containing structurally similar solutions, i.e., solutions with the same sparsity and sign pattern, over the whole range of parameters. Combining this with a model selection criterion, we can choose regularization parameters in a data-adaptive manner. Another advantage of our algorithm is that it provides an overview on the solution stability over the whole range of parameters. This can be further exploited to obtain additional insights into the problem of interest. We provide a numerical analysis of our method and compare it to the state-of-the-art single-penalty algorithms for compressed sensing problems in order to demonstrate the robustness and power of the proposed algorithm.

Recommendations

• Conditions on optimal support recovery in unmixing problems by means of multi-penalty regularization
• Minimization of multi-penalty functionals by alternating iterative thresholding and optimal parameter choices
• A nonconvex penalization algorithm with automatic choice of the regularization parameter in sparse imaging
• Adaptive regularization using the entire solution surface
• A new framework for multi-parameter regularization

Cites work

• A mathematical introduction to compressive sensing
• Conditions on optimal support recovery in unmixing problems by means of multi-penalty regularization
• Damping Noise-Folding and Enhanced Support Recovery in Compressed Sensing
• Greed is Good: Algorithmic Results for Sparse Approximation
• Hard thresholding pursuit algorithms: number of iterations
• Information Theoretic Bounds for Compressed Sensing
• Iterative hard thresholding for compressed sensing
• Least angle regression. (With discussion)
• Minimization of multi-penalty functionals by alternating iterative thresholding and optimal parameter choices
• Necessary and sufficient conditions for linear convergence of \(\ell^1\)-regularization
• On the consistency of feature selection using greedy least squares regression
• Oscillating patterns in image processing and nonlinear evolution equations. The fifteenth Dean Jacqueline B. Lewis memorial lectures
• Piecewise linear regularized solution paths
• Preconditioning the Lasso for sign consistency
• Regularization and Variable Selection Via the Elastic Net
• Robust Estimation of a Location Parameter
• Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $\ell _{1}$-Constrained Quadratic Programming (Lasso)
• Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit
• Sparse recovery via differential inclusions
• Sparsity-enforcing regularisation and ISTA revisited
• The Lasso problem and uniqueness
• Toeplitz and circulant matrices: a review.
• Uncertainty Principles and Signal Recovery
• Uncertainty principles and ideal atomic decomposition

Cited in

(5)

• Computational approaches to non-convex, sparsity-inducing multi-penalty regularization
• A review on the adaptive-ridge algorithm with several extensions
• Multi-parameter approaches in image processing
• L 1-Regularization Path Algorithm for Generalized Linear Models
• Conditions on optimal support recovery in unmixing problems by means of multi-penalty regularization

This page was built for publication: Adaptive multi-penalty regularization based on a generalized Lasso path

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2175012)