Simultaneous analysis of Lasso and Dantzig selector

DOI10.1214/08-AOS620MaRDI QIDQ2388978zbMATH OpenOpenAlexWikidataFDO

Authors Yaacov Ritov, Alexandre B. Tsybakov, P. J. Bickel

Publication date 22 July 2009

Published in The Annals of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/0801.1095

zbMATH Keywords

model selectionlinear modelsnonparametric statistics

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Asymptotic properties of nonparametric inference (62G20) Linear regression; mixed models (62J05) Prediction theory (aspects of stochastic processes) (60G25)

Abstract: We exhibit an approximate equivalence between the Lasso estimator and Dantzig selector. For both methods we derive parallel oracle inequalities for the prediction risk in the general nonparametric regression model, as well as bounds on the $el{l}_p$ estimation loss for $1leple2$ in the linear model when the number of variables can be much larger than the sample size.

Recommendations

• The oracle inequalities on simultaneous Lasso and Dantzig selector in high-dimensional nonparametric regression
• The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
• Rate minimaxity of the Lasso and Dantzig selector for the \(l_{q}\) loss in \(l_{r}\) balls
• The Dantzig selector and sparsity oracle inequalities
• Sparsity oracle inequalities for the Lasso

Cites work

• scientific article; zbMATH DE number 5957408 (Why is no real title available?)
• scientific article; zbMATH DE number 1522808 (Why is no real title available?)
• scientific article; zbMATH DE number 845714 (Why is no real title available?)
• A new approach to variable selection in least squares problems
• Aggregation and Sparsity Via ℓ1 Penalized Least Squares
• Aggregation for Gaussian regression
• Asymptotics for Lasso-type estimators.
• Functional aggregation for nonparametric regression.
• High-dimensional generalized linear models and the lasso
• High-dimensional graphs and variable selection with the Lasso
• Lasso-type recovery of sparse representations for high-dimensional data
• Least angle regression. (With discussion)
• Pathwise coordinate optimization
• Persistene in high-dimensional linear predictor-selection and the virtue of overparametrization
• Sparse Density Estimation with ℓ1 Penalties
• Sparsity in penalized empirical risk minimization
• Sparsity oracle inequalities for the Lasso
• Stable recovery of sparse overcomplete representations in the presence of noise
• The Dantzig selector and sparsity oracle inequalities
• The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
• The Group Lasso for Logistic Regression
• The sparsity and bias of the LASSO selection in high-dimensional linear regression

Cited in

(only showing first 100 items - show all)

• Approximate \(\ell_0\)-penalized estimation of piecewise-constant signals on graphs
• Adjusted regularized estimation in the accelerated failure time model with high dimensional covariates
• \(\alpha\)-variational inference with statistical guarantees
• Weighted l₁‐Penalized Corrected Quantile Regression for High‐Dimensional Temporally Dependent Measurement Errors
• A shrinkage principle for heavy-tailed data: high-dimensional robust low-rank matrix recovery
• On cross-validated Lasso in high dimensions
• Statistical inference in sparse high-dimensional additive models
• Fitting sparse linear models under the sufficient and necessary condition for model identification
• Inference for High-Dimensional Exchangeable Arrays
• A general theory of concave regularization for high-dimensional sparse estimation problems
• High-dimensional regression with unknown variance
• Adaptive estimation of the baseline hazard function in the Cox model by model selection, with high-dimensional covariates
• Variable selection and regularization via arbitrary rectangle-range generalized elastic net
• Aggregation of affine estimators
• The benefit of group sparsity
• Ridge regression and asymptotic minimax estimation over spheres of growing dimension
• Stable recovery of low rank matrices from nuclear norm minimization
• Comments on: \(\ell _{1}\)-penalization for mixture regression models
• Multiple change points detection in high-dimensional multivariate regression
• Sparse model uncertainties in compressed sensing with application to convolutions and sporadic communication
• High-Dimensional Gaussian Graphical Regression Models with Covariates
• Are discoveries spurious? Distributions of maximum spurious correlations and their applications
• Variable selection for high dimensional Gaussian copula regression model: an adaptive hypothesis testing procedure
• A Unified Framework for Change Point Detection in High-Dimensional Linear Models
• GenMod: a generative modeling approach for spectral representation of PDEs with random inputs
• Oracle efficient estimation of structural breaks in cointegrating regressions
• On estimation error bounds of the Elastic Net when p ≫ n
• A study on tuning parameter selection for the high-dimensional lasso
• Supervised Principal Component Regression for Functional Responses with High Dimensional Predictors
• A fast trans-lasso algorithm with penalized weighted score function
• Covariate adjustment in experiments with matched pairs
• Multi-block alternating direction method of multipliers for ultrahigh dimensional quantile fused regression
• Generalized nonconvex hyperspectral anomaly detection via background representation learning with dictionary constraint
• Locally adaptive sparse additive quantile regression model with TV penalty
• Matrix recovery from nonconvex regularized least absolute deviations
• Hierarchical inference for genome-wide association studies: a view on methodology with software
• Regularized 3D functional regression for brain image data via Haar wavelets
• Some sharp performance bounds for least squares regression with L₁ regularization
• Recovery of partly sparse and dense signals
• Estimation and variable selection in partial linear single index models with error-prone linear covariates
• Structured random measurements in signal processing
• Nearly optimal minimax estimator for high-dimensional sparse linear regression
• Grouping strategies and thresholding for high dimensional linear models
• A two-stage regularization method for variable selection and forecasting in high-order interaction model
• A systematic review on model selection in high-dimensional regression
• Detection boundary in sparse regression
• On the conditions used to prove oracle results for the Lasso
• The partial linear model in high dimensions
• Elastic-net Regularized High-dimensional Negative Binomial Regression: Consistency and Weak Signal Detection
• A statistical mechanics approach to de-biasing and uncertainty estimation in Lasso for random measurements
• Support vector machines with a reject option
• A Scale-Free Approach for False Discovery Rate Control in Generalized Linear Models
• Multivariate sparse Laplacian shrinkage for joint estimation of two graphical structures
• Solution of linear ill-posed problems using overcomplete dictionaries
• The variable selection by the Dantzig selector for Cox's proportional hazards model
• High dimensional regression for regenerative time-series: an application to road traffic modeling
• A reproducing kernel Hilbert space approach to high dimensional partially varying coefficient model
• Sparse recovery via nonconvex regularized \(M\)-estimators over \(\ell_q\)-balls
• Model selection for high-dimensional linear regression with dependent observations
• On Dantzig and Lasso estimators of the drift in a high dimensional Ornstein-Uhlenbeck model
• The horseshoe estimator: posterior concentration around nearly black vectors
• Posterior asymptotic normality for an individual coordinate in high-dimensional linear regression
• Adaptive Lasso and Dantzig selector for spatial point processes intensity estimation
• Overcoming the limitations of phase transition by higher order analysis of regularization techniques
• Finite-sample analysis of \(M\)-estimators using self-concordance
• The smooth-Lasso and other \(\ell _{1}+\ell _{2}\)-penalized methods
• Targeting underrepresented populations in precision medicine: a federated transfer learning approach
• LASSO Reloaded: A Variational Analysis Perspective with Applications to Compressed Sensing
• Convergence rates of variational posterior distributions
• Convergence of covariance and spectral density estimates for high-dimensional locally stationary processes
• Parallelism, uniqueness, and large-sample asymptotics for the Dantzig selector
• Sure independence screening for ultrahigh dimensional feature space. With discussion and authors' reply
• An _1-oracle inequality for the Lasso in multivariate finite mixture of multivariate Gaussian regression models
• Group sparse recovery via group square-root elastic net and the iterative multivariate thresholding-based algorithm
• Localized Gaussian width of \(M\)-convex hulls with applications to Lasso and convex aggregation
• Asymptotic risk and phase transition of \(l_1\)-penalized robust estimator
• Joint Structural Break Detection and Parameter Estimation in High-Dimensional Nonstationary VAR Models
• Sorted concave penalized regression
• Confidence intervals for low dimensional parameters in high dimensional linear models
• scientific article; zbMATH DE number 7626789 (Why is no real title available?)
• Regularized estimation in sparse high-dimensional multivariate regression, with application to a DNA methylation study
• Accuracy assessment for high-dimensional linear regression
• Adaptive Dantzig density estimation
• On verifiable sufficient conditions for sparse signal recovery via \(\ell_{1}\) minimization
• Nonconvex penalized ridge estimations for partially linear additive models in ultrahigh dimension
• Sharp oracle inequalities for square root regularization
• High dimensional change point estimation via sparse projection
• Group Lasso for structural break time series
• Least squares after model selection in high-dimensional sparse models
• Sign-constrained least squares estimation for high-dimensional regression
• Robust estimation for an inverse problem arising in multiview geometry
• Parallel integrative learning for large-scale multi-response regression with incomplete outcomes
• A global homogeneity test for high-dimensional linear regression
• Estimator selection: a new method with applications to kernel density estimation
• Forward-selected panel data approach for program evaluation
• Design of c-optimal experiments for high-dimensional linear models
• High dimensional generalized linear models for temporal dependent data
• Likelihood estimation of sparse topic distributions in topic models and its applications to Wasserstein document distance calculations
• On the use of cross-validation for the calibration of the adaptive Lasso
• Estimation of Linear Functionals in High-Dimensional Linear Models: From Sparsity to Nonsparsity

This page was built for publication: Simultaneous analysis of Lasso and Dantzig selector

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2388978)