On the distribution of the adaptive LASSO estimator

DOI10.1016/J.JSPI.2009.01.003MaRDI QIDQ1022011zbMATH OpenOpenAlexFDO

Authors Ulrike Schneider, Benedikt M. Pötscher

Publication date 9 June 2009

Published in Journal of Statistical Planning and Inference (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/0801.4627

zbMATH Keywords

penalized maximum likelihoodasymptotic distributionoracle propertyLASSOuniform consistencyadaptive LASSOestimation of distributionsfinite-sample distributionnonnegative garotte

Mathematics Subject Classification ID

Exact distribution theory in statistics (62E15) Asymptotic properties of parametric estimators (62F12) Linear regression; mixed models (62J05) Monte Carlo methods (65C05) Asymptotic distribution theory in statistics (62E20) Ridge regression; shrinkage estimators (Lasso) (62J07)

Abstract: We study the distribution of the adaptive LASSO estimator (Zou (2006)) in finite samples as well as in the large-sample limit. The large-sample distributions are derived both for the case where the adaptive LASSO estimator is tuned to perform conservative model selection as well as for the case where the tuning results in consistent model selection. We show that the finite-sample as well as the large-sample distributions are typically highly non-normal, regardless of the choice of the tuning parameter. The uniform convergence rate is also obtained, and is shown to be slower than $n^{-1/2}$ in case the estimator is tuned to perform consistent model selection. In particular, these results question the statistical relevance of the `oracle' property of the adaptive LASSO estimator established in Zou (2006). Moreover, we also provide an impossibility result regarding the estimation of the distribution function of the adaptive LASSO estimator.The theoretical results, which are obtained for a regression model with orthogonal design, are complemented by a Monte Carlo study using non-orthogonal regressors.

Recommendations

• On the distribution of penalized maximum likelihood estimators: the LASSO, SCAD, and thresholding
• Adaptive Lasso in high-dimensional settings
• Rates of convergence of the adaptive LASSO estimators to the oracle distribution and higher order refinements by the bootstrap
• Adaptive Lasso for sparse high-dimensional regression models
• Adaptive lasso for generalized linear models with a diverging number of parameters

Cites work

• scientific article; zbMATH DE number 845714 (Why is no real title available?)
• A Statistical View of Some Chemometrics Regression Tools
• A note on the Lasso and related procedures in model selection
• Adaptive Lasso for Cox's proportional hazards model
• Asymptotics for Lasso-type estimators.
• Better Subset Regression Using the Nonnegative Garrote
• CAN ONE ESTIMATE THE UNCONDITIONAL DISTRIBUTION OF POST-MODEL-SELECTION ESTIMATORS?
• Can one estimate the conditional distribution of post-model-selection estimators?
• Can the strengths of AIC and BIC be shared? A conflict between model indentification and regression estimation
• Composite quantile regression and the oracle model selection theory
• Consistent covariate selection and post model selection inference in semiparametric regression.
• Covariate selection for semiparametric hazard function regression models
• Least angle regression. (With discussion)
• MODEL SELECTION AND INFERENCE: FACTS AND FICTION
• Model selection and estimation in the Gaussian graphical model
• Model selection by multiple test procedures
• New Estimation and Model Selection Procedures for Semiparametric Modeling in Longitudinal Data Analysis
• One-step sparse estimates in nonconcave penalized likelihood models
• PERFORMANCE LIMITS FOR ESTIMATORS OF THE RISK OR DISTRIBUTION OF SHRINKAGE-TYPE ESTIMATORS, AND SOME GENERAL LOWER RISK-BOUND RESULTS
• Penalized Estimating Functions and Variable Selection in Semiparametric Regression Models
• Regression coefficient and autoregressive order shrinkage and selection via the lasso
• Sparse estimators and the oracle property, or the return of Hodges' estimator
• THE FINITE-SAMPLE DISTRIBUTION OF POST-MODEL-SELECTION ESTIMATORS AND UNIFORM VERSUS NONUNIFORM APPROXIMATIONS
• The Adaptive Lasso and Its Oracle Properties
• Tuning parameter selectors for the smoothly clipped absolute deviation method
• Unified LASSO Estimation by Least Squares Approximation
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
• Variable selection for Cox's proportional hazards model and frailty model
• Variable selection in semiparametric regression modeling

Cited in

(33)

• Shrinkage estimation of regression models with multiple structural changes
• The Lasso estimator: distributional properties.
• On the residual empirical process based on the ALASSO in high dimensions and its functional oracle property
• Penalized maximum likelihood estimation of a stochastic multivariate regression model
• Confidence sets based on thresholding estimators in high-dimensional Gaussian regression models
• The Lasso on latent indices for regression modeling with ordinal categorical predictors
• Model selection and inference for censored lifetime medical expenditures
• A relative error-based approach for variable selection
• Consistent and conservative model selection with the adaptive Lasso in stationary and nonstationary autoregressions
• The distribution of the Lasso: uniform control over sparse balls and adaptive parameter tuning
• Variable selection using P-splines
• Shrinkage estimation of dynamic panel data models with interactive fixed effects
• UNIFORM ASYMPTOTICS AND CONFIDENCE REGIONS BASED ON THE ADAPTIVE LASSO WITH PARTIALLY CONSISTENT TUNING
• Valid post-selection inference
• Rates of convergence of the adaptive LASSO estimators to the oracle distribution and higher order refinements by the bootstrap
• The risk of James-Stein and Lasso shrinkage
• Monte Carlo simulation for Lasso-type problems by estimator augmentation
• On various confidence intervals post-model-selection
• Model selection by LASSO methods in a change-point model
• On the impact of model selection on predictor identification and parameter inference
• Multiple structural breaks in cointegrating regressions: a model selection approach
• Adaptive LASSO-type estimation for multivariate diffusion processes
• A note on the asymptotic distribution of lasso estimator for correlated data
• On the distribution of penalized maximum likelihood estimators: the LASSO, SCAD, and thresholding
• Oracle properties, bias correction, and bootstrap inference for adaptive lasso for time series \(M\)-estimators
• Asymptotic properties of Lasso+mLS and Lasso+Ridge in sparse high-dimensional linear regression
• Asymptotic linear expansion of regularized M-estimators
• On Hodges' superefficiency and merits of oracle property in model selection
• On the distribution, model selection properties and uniqueness of the Lasso estimator in low and high dimensions
• D-trace estimation of a precision matrix using adaptive lasso penalties
• Confidence sets based on penalized maximum likelihood estimators in Gaussian regression
• Distributional results for thresholding estimators in high-dimensional Gaussian regression models
• Estimation and variable selection in partial linear single index models with error-prone linear covariates

This page was built for publication: On the distribution of the adaptive LASSO estimator

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1022011)