On the distribution of the adaptive LASSO estimator

DOI10.1016/J.JSPI.2009.01.003zbMATH Open1162.62063arXiv0801.4627OpenAlexW2131679483MaRDI QIDQ1022011FDOQ1022011

Authors: Ulrike Schneider, Benedikt M. Pötscher 

Publication date: 9 June 2009

Published in: Journal of Statistical Planning and Inference (Search for Journal in Brave)

Abstract: We study the distribution of the adaptive LASSO estimator (Zou (2006)) in finite samples as well as in the large-sample limit. The large-sample distributions are derived both for the case where the adaptive LASSO estimator is tuned to perform conservative model selection as well as for the case where the tuning results in consistent model selection. We show that the finite-sample as well as the large-sample distributions are typically highly non-normal, regardless of the choice of the tuning parameter. The uniform convergence rate is also obtained, and is shown to be slower than $n^{-1/2}$ in case the estimator is tuned to perform consistent model selection. In particular, these results question the statistical relevance of the `oracle' property of the adaptive LASSO estimator established in Zou (2006). Moreover, we also provide an impossibility result regarding the estimation of the distribution function of the adaptive LASSO estimator.The theoretical results, which are obtained for a regression model with orthogonal design, are complemented by a Monte Carlo study using non-orthogonal regressors.

Full work available at URL: https://arxiv.org/abs/0801.4627

Recommendations

• On the distribution of penalized maximum likelihood estimators: the LASSO, SCAD, and thresholding
• Adaptive Lasso in high-dimensional settings
• Rates of convergence of the adaptive LASSO estimators to the oracle distribution and higher order refinements by the bootstrap
• Adaptive Lasso for sparse high-dimensional regression models
• Adaptive lasso for generalized linear models with a diverging number of parameters

zbMATH Keywords

penalized maximum likelihoodasymptotic distributionoracle propertyLASSOuniform consistencyadaptive LASSOestimation of distributionsfinite-sample distributionnonnegative garotte

Mathematics Subject Classification ID

Exact distribution theory in statistics (62E15) Asymptotic properties of parametric estimators (62F12) Linear regression; mixed models (62J05) Monte Carlo methods (65C05) Asymptotic distribution theory in statistics (62E20) Ridge regression; shrinkage estimators (Lasso) (62J07)

Cites Work

• The Adaptive Lasso and Its Oracle Properties
• Least angle regression. (With discussion)
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
• Title not available (Why is that?)
• One-step sparse estimates in nonconcave penalized likelihood models
• Asymptotics for Lasso-type estimators.
• Variable selection for Cox's proportional hazards model and frailty model
• Sparse estimators and the oracle property, or the return of Hodges' estimator
• Model selection and estimation in the Gaussian graphical model
• Can the strengths of AIC and BIC be shared? A conflict between model indentification and regression estimation
• Unified LASSO Estimation by Least Squares Approximation
• Regression coefficient and autoregressive order shrinkage and selection via the lasso
• Tuning parameter selectors for the smoothly clipped absolute deviation method
• MODEL SELECTION AND INFERENCE: FACTS AND FICTION
• New Estimation and Model Selection Procedures for Semiparametric Modeling in Longitudinal Data Analysis
• Variable selection in semiparametric regression modeling
• Better Subset Regression Using the Nonnegative Garrote
• A Statistical View of Some Chemometrics Regression Tools
• Can one estimate the conditional distribution of post-model-selection estimators?
• Composite quantile regression and the oracle model selection theory
• A note on the Lasso and related procedures in model selection
• Adaptive Lasso for Cox's proportional hazards model
• PERFORMANCE LIMITS FOR ESTIMATORS OF THE RISK OR DISTRIBUTION OF SHRINKAGE-TYPE ESTIMATORS, AND SOME GENERAL LOWER RISK-BOUND RESULTS
• CAN ONE ESTIMATE THE UNCONDITIONAL DISTRIBUTION OF POST-MODEL-SELECTION ESTIMATORS?
• THE FINITE-SAMPLE DISTRIBUTION OF POST-MODEL-SELECTION ESTIMATORS AND UNIFORM VERSUS NONUNIFORM APPROXIMATIONS
• Penalized Estimating Functions and Variable Selection in Semiparametric Regression Models
• Consistent covariate selection and post model selection inference in semiparametric regression.
• Covariate selection for semiparametric hazard function regression models
• Model selection by multiple test procedures

Cited In (29)

• Shrinkage estimation of dynamic panel data models with interactive fixed effects
• On various confidence intervals post-model-selection
• Estimation and variable selection in partial linear single index models with error-prone linear covariates
• The LASSO on latent indices for regression modeling with ordinal categorical predictors
• Penalized maximum likelihood estimation of a stochastic multivariate regression model
• UNIFORM ASYMPTOTICS AND CONFIDENCE REGIONS BASED ON THE ADAPTIVE LASSO WITH PARTIALLY CONSISTENT TUNING
• On Hodges' superefficiency and merits of oracle property in model selection
• Model selection by LASSO methods in a change-point model
• On the distribution, model selection properties and uniqueness of the Lasso estimator in low and high dimensions
• Multiple structural breaks in cointegrating regressions: a model selection approach
• Variable selection using P-splines
• On the impact of model selection on predictor identification and parameter inference
• Confidence sets based on penalized maximum likelihood estimators in Gaussian regression
• Distributional results for thresholding estimators in high-dimensional Gaussian regression models
• On the residual empirical process based on the ALASSO in high dimensions and its functional oracle property
• Consistent and conservative model selection with the adaptive Lasso in stationary and nonstationary autoregressions
• Monte Carlo Simulation for Lasso-Type Problems by Estimator Augmentation
• Confidence Sets Based on Thresholding Estimators in High-Dimensional Gaussian Regression Models
• A relative error-based approach for variable selection
• SHRINKAGE ESTIMATION OF REGRESSION MODELS WITH MULTIPLE STRUCTURAL CHANGES
• Asymptotic linear expansion of regularized M-estimators
• Adaptive LASSO-type estimation for multivariate diffusion processes
• Valid post-selection inference
• Rates of convergence of the adaptive LASSO estimators to the oracle distribution and higher order refinements by the bootstrap
• D-trace estimation of a precision matrix using adaptive lasso penalties
• Model selection and inference for censored lifetime medical expenditures
• On the distribution of penalized maximum likelihood estimators: the LASSO, SCAD, and thresholding
• The Risk of James–Stein and Lasso Shrinkage
• Asymptotic properties of Lasso+mLS and Lasso+Ridge in sparse high-dimensional linear regression

This page was built for publication: On the distribution of the adaptive LASSO estimator

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1022011)