Within group variable selection through the exclusive Lasso

DOI10.1214/17-EJS1317MaRDI QIDQ1676787zbMATH OpenFDO

Authors Frederick Campbell, Genevera I. Allen

Publication date 10 November 2017

Published in Electronic Journal of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1505.07517, https://projecteuclid.org/euclid.ejs/1509004863

zbMATH Keywords

composite penaltyexclusive LassoNMR spectroscopystructured variable selection

Mathematics Subject Classification ID

Ridge regression; shrinkage estimators (Lasso) (62J07) Applications of statistics to physics (62P35)

Abstract: Many data sets consist of variables with an inherent group structure. The problem of group selection has been well studied, but in this paper, we seek to do the opposite: our goal is to select at least one variable from each group in the context of predictive regression modeling. This problem is NP-hard, but we study the tightest convex relaxation: a composite penalty that is a combination of the $el{l}_1$ and $el{l}_2$ norms. Our so-called Exclusive Lasso method performs structured variable selection by ensuring that at least one variable is selected from each group. We study our method's statistical properties and develop computationally scalable algorithms for fitting the Exclusive Lasso. We study the effectiveness of our method via simulations as well as using NMR spectroscopy data. Here, we use the Exclusive Lasso to select the appropriate chemical shift from a dictionary of possible chemical shifts for each molecule in the biological sample.

Review text

The authors of the paper under review consider regression problems with a predefined group structure to the end of accurately predicting the response using a subset of variables consisting of at least one variable from each group. In order to ``directly enforce the desired structure they present a ``methodology for sparse regression with the exclusive Lasso, a convex penalty first introduced by \textit{Y. Zhou} et al. [``Exclusive Lasso for multi-task feature selection, Proc. Mach. Learn. Res. (PMLR) 9, 988--995 (2010)] in the context of multi-task learning. Next, statistical properties of the introduced methodology are studied and two algorithms for fitting the exclusive Lasso are provided. The effectiveness of the methodology is then studied via simulations as well as using Nuclear Magnetic Resonance spectroscopy data.

Recommendations

• The group exponential Lasso for bi-level variable selection
• Consistent group selection in high-dimensional linear regression
• A selective review of group selection in high-dimensional models
• The Group Lasso for Logistic Regression
• Group variable selection via a hierarchical lasso and its oracle property

Cited in

(16)

• Sparse classification with paired covariates
• Sparse regression for extreme values
• Biclustering via structured regularized matrix decomposition
• Group variable selection via \(\ell_{p,0}\) regularization and application to optimal scoring
• Group variable selection via a hierarchical lasso and its oracle property
• Purely time-dependent optimal control of quasilinear parabolic PDEs with sparsity enforcing penalization
• Detection of similar successive groups in a model with diverging number of variable groups
• Group variable selection via SCAD-L₂
• Variable selection, monotone likelihood ratio and group sparsity
• Variance estimation in high-dimensional linear regression via adaptive elastic-net
• A highly efficient algorithm for solving exclusive lasso problems
• A Newton-based variant of exclusive Lasso for improved sparse solutions
• Plus-minus models for evaluating and scouting soccer players using possession sequences
• Adaptive sieving: a dimension reduction technique for sparse optimization problems
• Optimizing variable selection in multi-omics datasets: a focus on exclusive Lasso
• Group regression with tailored penalties for high-dimensional data with or without predefined groups

This page was built for publication: Within group variable selection through the exclusive Lasso

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1676787)