The Lasso, correlated design, and improved oracle inequalities

DOI10.1214/12-IMSCOLL922MaRDI QIDQ5499696zbMATH OpenFDO

Authors Sara Van De Geer, Johannes Lederer

Publication date 30 July 2015

Published in Institute of Mathematical Statistics Collections (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1107.0189

zbMATH Keywords

compatibilitycorrelationentropyhigh-dimensional modelLasso

Mathematics Subject Classification ID

Linear regression; mixed models (62J05) Ridge regression; shrinkage estimators (Lasso) (62J07) Linear inference, regression (62J99)

Abstract: We study high-dimensional linear models and the $el{l}_1$-penalized least squares estimator, also known as the Lasso estimator. In literature, oracle inequalities have been derived under restricted eigenvalue or compatibility conditions. In this paper, we complement this with entropy conditions which allow one to improve the dual norm bound, and demonstrate how this leads to new oracle inequalities. The new oracle inequalities show that a smaller choice for the tuning parameter and a trade-off between $el{l}_1$-norms and small compatibility constants are possible. This implies, in particular for correlated design, improved bounds for the prediction error of the Lasso estimator as compared to the methods based on restricted eigenvalue or compatibility conditions only.

Recommendations

• Lasso, iterative feature selection and the correlation selector: oracle inequalities and numerical performances
• The Lasso with general Gaussian designs with applications to hypothesis testing
• Oracle inequalities for the Lasso in the high-dimensional Aalen multiplicative intensity model
• Sparsity oracle inequalities for the Lasso
• scientific article; zbMATH DE number 1850463
• A note on the asymptotic distribution of lasso estimator for correlated data
• Slope meets Lasso: improved oracle bounds and optimality
• On the conditions used to prove oracle results for the Lasso
• Oracle inequalities for the lasso in the Cox model
• The Adaptive Lasso and Its Oracle Properties

Cited in

(23)

• Oracle inequalities for high-dimensional prediction
• Lasso, iterative feature selection and the correlation selector: oracle inequalities and numerical performances
• On the conditions used to prove oracle results for the Lasso
• The Lasso as an \(\ell _{1}\)-ball model selection procedure
• Slope meets Lasso: improved oracle bounds and optimality
• Aggregated hold out for sparse linear regression with a robust loss function
• Inference for high-dimensional instrumental variables regression
• Prediction error bounds for linear regression with the TREX
• Tuning parameter calibration for _1-regularized logistic regression
• Leave-one-out cross-validation is risk consistent for Lasso
• Restricted eigenvalue properties for correlated Gaussian designs
• Upper bound on the estimation error in a growth curve model
• Oracle inequalities for the lasso in the Cox model
• Correlated variables in regression: clustering and sparse estimation
• On tight bounds for the Lasso
• Sharp oracle inequalities for square root regularization
• A new perspective on least squares under convex constraint
• On the prediction performance of the Lasso
• Finite-sample results for lasso and stepwise Neyman-orthogonal Poisson estimators
• scientific article; zbMATH DE number 7626791 (Why is no real title available?)
• Binarsity: a penalization for one-hot encoded features in linear supervised learning
• On non-asymptotic bounds for estimation in generalized linear models with highly correlated design
• Quasi-likelihood and/or robust estimation in high dimensions

This page was built for publication: The Lasso, correlated design, and improved oracle inequalities

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5499696)