Sparse estimators and the oracle property, or the return of Hodges' estimator

DOI10.1016/J.JECONOM.2007.05.017MaRDI QIDQ290948zbMATH OpenOpenAlexFDO

Authors Hannes Leeb, Benedikt M. Pötscher

Publication date 3 June 2016

Published in Journal of Econometrics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/0704.1466

zbMATH Keywords

LassoSCADpenalized maximum likelihoodsparsityoracle propertybridge estimatorhard-thresholdingHodges' estimatormaximal absolute biasmaximal risknonuniform limitspenalized least squares

Mathematics Subject Classification ID

Asymptotic properties of parametric estimators (62F12) Nonparametric regression and quantile regression (62G08) Linear regression; mixed models (62J05) Estimation in survival analysis and censored data (62N02) Applications of statistics to economics (62P20) Ridge regression; shrinkage estimators (Lasso) (62J07)

Abstract: We point out some pitfalls related to the concept of an oracle property as used in Fan and Li (2001, 2002, 2004) which are reminiscent of the well-known pitfalls related to Hodges' estimator. The oracle property is often a consequence of sparsity of an estimator. We show that any estimator satisfying a sparsity property has maximal risk that converges to the supremum of the loss function; in particular, the maximal risk diverges to infinity whenever the loss function is unbounded. For ease of presentation the result is set in the framework of a linear regression model, but generalizes far beyond that setting. In a Monte Carlo study we also assess the extent of the problem in finite samples for the smoothly clipped absolute deviation (SCAD) estimator introduced in Fan and Li (2001). We find that this estimator can perform rather poorly in finite samples and that its worst-case performance relative to maximum likelihood deteriorates with increasing sample size when the estimator is tuned to sparsity.

Recommendations

• On Hodges' superefficiency and merits of oracle property in model selection
• Smoothly clipped absolute deviation on high dimensions
• Sparsity oracle inequalities for the Lasso
• Asymptotic oracle properties of SCAD-penalized least squares estimators
• Large sample properties of the SCAD-penalized maximum likelihood estimation on high dimen\-sions

Cites work

• scientific article; zbMATH DE number 3886925 (Why is no real title available?)
• scientific article; zbMATH DE number 3899978 (Why is no real title available?)
• scientific article; zbMATH DE number 3942850 (Why is no real title available?)
• scientific article; zbMATH DE number 4084766 (Why is no real title available?)
• scientific article; zbMATH DE number 3614055 (Why is no real title available?)
• scientific article; zbMATH DE number 1220667 (Why is no real title available?)
• scientific article; zbMATH DE number 845714 (Why is no real title available?)
• scientific article; zbMATH DE number 3258670 (Why is no real title available?)
• A Statistical View of Some Chemometrics Regression Tools
• Asymptotics for Lasso-type estimators.
• Can the strengths of AIC and BIC be shared? A conflict between model indentification and regression estimation
• Consistent covariate selection and post model selection inference in semiparametric regression.
• Covariate selection for semiparametric hazard function regression models
• MODEL SELECTION AND INFERENCE: FACTS AND FICTION
• New Estimation and Model Selection Procedures for Semiparametric Modeling in Longitudinal Data Analysis
• Nonconcave penalized likelihood with a diverging number of parameters.
• ON VARIABLE SELECTION IN LINEAR REGRESSION
• PERFORMANCE LIMITS FOR ESTIMATORS OF THE RISK OR DISTRIBUTION OF SHRINKAGE-TYPE ESTIMATORS, AND SOME GENERAL LOWER RISK-BOUND RESULTS
• Selection of the number of regression variables; A minimax choice of generalized FPE
• The Adaptive Lasso and Its Oracle Properties
• The risk inflation criterion for multiple regression
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
• Variable selection for Cox's proportional hazards model and frailty model
• Variable selection for multivariate failure time data

Cited in

(62)

• Sparse generalized Yule-Walker estimation for large spatio-temporal autoregressions with an application to NO\(_2\) satellite data
• Information criteria for model selection
• Confidence intervals for intentionally biased estimators
• An averaging estimator for two-step m-estimation in semiparametric models
• An alternative to synthetic control for models with many covariates under sparsity
• scientific article; zbMATH DE number 7626707 (Why is no real title available?)
• A PANEL CLUSTERING APPROACH TO ANALYZING BUBBLE BEHAVIOR
• Estimation of Sparsity-Induced Weak Factor Models
• Inference in Sparsity-Induced Weak Factor Models
• Shrinkage estimation of regression models with multiple structural changes
• Jump estimation in inverse regression
• On the residual empirical process based on the ALASSO in high dimensions and its functional oracle property
• A Bayesian approach to sparse dynamic network identification
• Robust inference on average treatment effects with possibly more covariates than observations
• Automated estimation of vector error correction models
• Moderately clipped Lasso
• Bridge estimators and the adaptive Lasso under heteroscedasticity
• \(\ell_1\)-regularization of high-dimensional time-series models with non-Gaussian and heteroskedastic errors
• Uniformly valid post-regularization confidence regions for many functional parameters in z-estimation framework
• Simultaneous multiple non-crossing quantile regression estimation using kernel constraints
• Covariate selection with group Lasso and doubly robust estimation of causal effects
• Exact adaptive confidence intervals for linear regression coefficients
• Comments on: \(\ell_{1}\)-penalization for mixture regression models
• On the asymptotic variance of the debiased Lasso
• Quantile-based portfolios: post-model-selection estimation with alternative specifications
• Finite-sample results for lasso and stepwise Neyman-orthogonal Poisson estimators
• Shrinkage estimation of dynamic panel data models with interactive fixed effects
• Testing Sparsity-Inducing Penalties
• Select the valid and relevant moments: an information-based Lasso for GMM with many moments
• The adaptive Lasso in high-dimensional sparse heteroscedastic models
• Quantifying the cost of simultaneous non-parametric approximation of several samples
• Valid post-selection inference
• Asymptotically efficient estimators for stochastic blockmodels: the naive MLE, the rank-constrained MLE, and the spectral estimator
• Lassoing the determinants of retirement
• Estimating heterogeneous graphical models for discrete data with an application to roll call voting
• Shrinkage for categorical regressors
• The costs and benefits of uniformly valid causal inference with high-dimensional nuisance parameters
• ArCo: an artificial counterfactual approach for high-dimensional panel time-series data
• Lasso Inference for High-Dimensional Time Series
• The risk of James-Stein and Lasso shrinkage
• Using invalid instruments on purpose: focused moment selection and averaging for GMM
• Asymptotic properties of the residual bootstrap for lasso estimators
• The GENIUS approach to robust Mendelian randomization inference
• Constrained estimation using penalization and MCMC
• The robust desparsified lasso and the focused information criterion for high-dimensional generalized linear models
• Evaluation and selection of models for out-of-sample prediction when the sample size is small relative to the complexity of the data-generating process
• Valid post-selection inference in high-dimensional approximately sparse quantile regression models
• On the distribution of penalized maximum likelihood estimators: the LASSO, SCAD, and thresholding
• On Hodges' superefficiency and merits of oracle property in model selection
• Adaptive GMM shrinkage estimation with consistent moment selection
• Panel data quantile regression with grouped fixed effects
• On the distribution of the adaptive LASSO estimator
• Endogeneity in high dimensions
• Nonconcave penalized inverse regression in single-index models with high dimensional predic\-tors
• Consistencies and rates of convergence of jump-penalized least squares estimators
• Generalized additive partial linear models with high-dimensional covariates
• Sparse estimation from noisy observations of an overdetermined linear system
• A necessary condition for the strong oracle property
• Confidence sets based on penalized maximum likelihood estimators in Gaussian regression
• Distributional results for thresholding estimators in high-dimensional Gaussian regression models
• A robust test for network generated dependence
• Statistical inference for normal mixtures with unknown number of components

This page was built for publication: Sparse estimators and the oracle property, or the return of Hodges' estimator

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q290948)