Regularizers for structured sparsity
From MaRDI portal
Publication:1949299
Abstract: We study the problem of learning a sparse linear regression vector under additional conditions on the structure of its sparsity pattern. This problem is relevant in machine learning, statistics and signal processing. It is well known that a linear regression can benefit from knowledge that the underlying regression vector is sparse. The combinatorial problem of selecting the nonzero components of this vector can be "relaxed" by regularizing the squared error with a convex penalty function like the norm. However, in many applications, additional conditions on the structure of the regression vector and its sparsity pattern are available. Incorporating this information into the learning method may lead to a significant decrease of the estimation error. In this paper, we present a family of convex penalty functions, which encode prior knowledge on the structure of the vector formed by the absolute values of the regression coefficients. This family subsumes the norm and is flexible enough to include different models of sparsity patterns, which are of practical and theoretical importance. We establish the basic properties of these penalty functions and discuss some examples where they can be computed explicitly. Moreover, we present a convergent optimization algorithm for solving regularized least squares with these penalty functions. Numerical simulations highlight the benefit of structured sparsity and the advantage offered by our approach over the Lasso method and other related methods.
Recommendations
Cites work
- Convex multi-task feature learning
- Feature space perspectives for learning the kernel
- High-dimensional generalized linear models and the lasso
- scientific article; zbMATH DE number 5968915 (Why is no real title available?)
- scientific article; zbMATH DE number 1818892 (Why is no real title available?)
- scientific article; zbMATH DE number 2107836 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- scientific article; zbMATH DE number 5251637 (Why is no real title available?)
- Learning the kernel matrix with semidefinite programming
- Learning with Structured Sparsity
- Model Selection and Estimation in Regression with Grouped Variables
- Nonparametric sparsity and regularization
- On spectral learning
- Oracle inequalities and optimal inference under group sparsity
- Simultaneous analysis of Lasso and Dantzig selector
- Sparsity in multiple kernel learning
- Sparsity oracle inequalities for the Lasso
- Structured variable selection and estimation
- Structured variable selection with sparsity-inducing norms
- Sup-norm convergence rate and sign concentration property of Lasso and Dantzig estimators
- The composite absolute penalties family for grouped and hierarchical variable selection
- The Theory of Max-Min, with Applications
Cited in
(39)- Sparsity and persistence: mixed norms provide simple signal models with dependent coefficients
- Two-level structural sparsity regularization for identifying lattices and defects in noisy images
- Perspective functions: properties, constructions, and examples
- A note on ``Reguralizers for structured sparsity
- Pairwise fusion approach incorporating prior constraint information
- Structured iterative hard thresholding with on- and off-grid applications
- Sparse regularization via bidualization
- Structured sparsity promoting functions
- Perspective maximum likelihood-type estimation via proximal decomposition
- Sparse learning via Boolean relaxations
- Learning with optimal interpolation norms
- Sparsity-enforcing regularisation and ISTA revisited
- scientific article; zbMATH DE number 6378171 (Why is no real title available?)
- Structured sparsity: discrete and convex approaches
- Smoothing proximal gradient method for general structured sparse regression
- Learning the Structure for Structured Sparsity
- Variational Gram functions: convex analysis and optimization
- Sparse Regularization via Convex Analysis
- Sharp oracle inequalities for square root regularization
- Generalized conditional gradient for sparse estimation
- Fast algorithms for structured sparsity (ICALP 2015 invited tutorial)
- Sparsity method for network structure of broad learning system based on lasso and elastic net
- scientific article; zbMATH DE number 7626791 (Why is no real title available?)
- A First-Order Optimization Algorithm for Statistical Learning with Hierarchical Sparsity Structure
- Binarsity: a penalization for one-hot encoded features in linear supervised learning
- On sparsity inducing regularization methods for machine learning
- Structured variable selection with sparsity-inducing norms
- Learning with Structured Sparsity
- Structured sparsity and generalization
- Structured sparsity via alternating direction methods
- Weakly decomposable regularization penalties and structured sparsity
- Structured sparsity through convex optimization
- Smooth over-parameterized solvers for non-smooth structured optimization
- Enhanced computation of the proximity operator for perspective functions
- The geometry of monotone operator splitting methods
- Perspective functions with nonlinear scaling
- The structured elastic net for quantile regression and support vector classification
- Convex relaxations of penalties for sparse correlated variables with bounded total variation
- Regularized vector field learning with sparse approximation for mismatch removal
This page was built for publication: Regularizers for structured sparsity
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1949299)
