Sparse estimators and the oracle property, or the return of Hodges' estimator

DOI10.1016/J.JECONOM.2007.05.017zbMATH Open1418.62272arXiv0704.1466OpenAlexW2019473361MaRDI QIDQ290948FDOQ290948

Hannes Leeb, Benedikt M. Pötscher

Publication date: 3 June 2016

Published in: Journal of Econometrics (Search for Journal in Brave)

Abstract: We point out some pitfalls related to the concept of an oracle property as used in Fan and Li (2001, 2002, 2004) which are reminiscent of the well-known pitfalls related to Hodges' estimator. The oracle property is often a consequence of sparsity of an estimator. We show that any estimator satisfying a sparsity property has maximal risk that converges to the supremum of the loss function; in particular, the maximal risk diverges to infinity whenever the loss function is unbounded. For ease of presentation the result is set in the framework of a linear regression model, but generalizes far beyond that setting. In a Monte Carlo study we also assess the extent of the problem in finite samples for the smoothly clipped absolute deviation (SCAD) estimator introduced in Fan and Li (2001). We find that this estimator can perform rather poorly in finite samples and that its worst-case performance relative to maximum likelihood deteriorates with increasing sample size when the estimator is tuned to sparsity.

Full work available at URL: https://arxiv.org/abs/0704.1466

Recommendations

• On Hodges' superefficiency and merits of oracle property in model selection
• Smoothly clipped absolute deviation on high dimensions
• Sparsity oracle inequalities for the Lasso
• Asymptotic oracle properties of SCAD-penalized least squares estimators
• Large sample properties of the SCAD-penalized maximum likelihood estimation on high dimen\-sions

zbMATH Keywords

LassoSCADpenalized maximum likelihoodsparsityoracle propertybridge estimatorhard-thresholdingHodges' estimatormaximal absolute biasmaximal risknonuniform limitspenalized least squares

Mathematics Subject Classification ID

Asymptotic properties of parametric estimators (62F12) Nonparametric regression and quantile regression (62G08) Linear regression; mixed models (62J05) Estimation in survival analysis and censored data (62N02) Applications of statistics to economics (62P20) Ridge regression; shrinkage estimators (Lasso) (62J07)

Cites Work

• The Adaptive Lasso and Its Oracle Properties
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Asymptotics for Lasso-type estimators.
• Variable selection for Cox's proportional hazards model and frailty model
• Can the strengths of AIC and BIC be shared? A conflict between model indentification and regression estimation
• Title not available (Why is that?)
• MODEL SELECTION AND INFERENCE: FACTS AND FICTION
• New Estimation and Model Selection Procedures for Semiparametric Modeling in Longitudinal Data Analysis
• Nonconcave penalized likelihood with a diverging number of parameters.
• The risk inflation criterion for multiple regression
• Title not available (Why is that?)
• A Statistical View of Some Chemometrics Regression Tools
• PERFORMANCE LIMITS FOR ESTIMATORS OF THE RISK OR DISTRIBUTION OF SHRINKAGE-TYPE ESTIMATORS, AND SOME GENERAL LOWER RISK-BOUND RESULTS
• Title not available (Why is that?)
• Consistent covariate selection and post model selection inference in semiparametric regression.
• Selection of the number of regression variables; A minimax choice of generalized FPE
• Covariate selection for semiparametric hazard function regression models
• Title not available (Why is that?)
• Title not available (Why is that?)
• ON VARIABLE SELECTION IN LINEAR REGRESSION
• Variable selection for multivariate failure time data

Cited In (61)

• Sparse generalized Yule-Walker estimation for large spatio-temporal autoregressions with an application to NO\(_2\) satellite data
• Confidence intervals for intentionally biased estimators
• Estimation of Sparsity-Induced Weak Factor Models
• Inference in Sparsity-Induced Weak Factor Models
• Information criteria for model selection
• An alternative to synthetic control for models with many covariates under sparsity
• An averaging estimator for two-step m-estimation in semiparametric models
• A PANEL CLUSTERING APPROACH TO ANALYZING BUBBLE BEHAVIOR
• Shrinkage estimation of dynamic panel data models with interactive fixed effects
• The adaptive Lasso in high-dimensional sparse heteroscedastic models
• On Hodges' superefficiency and merits of oracle property in model selection
• Panel data quantile regression with grouped fixed effects
• Bridge estimators and the adaptive Lasso under heteroscedasticity
• The costs and benefits of uniformly valid causal inference with high-dimensional nuisance parameters
• Evaluation and selection of models for out-of-sample prediction when the sample size is small relative to the complexity of the data-generating process
• On the distribution of the adaptive LASSO estimator
• On the asymptotic variance of the debiased Lasso
• Using invalid instruments on purpose: focused moment selection and averaging for GMM
• Nonconcave penalized inverse regression in single-index models with high dimensional predic\-tors
• Uniformly valid post-regularization confidence regions for many functional parameters in z-estimation framework
• GENERALIZED ADDITIVE PARTIAL LINEAR MODELS WITH HIGH-DIMENSIONAL COVARIATES
• Endogeneity in high dimensions
• Constrained estimation using penalization and MCMC
• Asymptotically efficient estimators for stochastic blockmodels: the naive MLE, the rank-constrained MLE, and the spectral estimator
• Shrinkage for categorical regressors
• Statistical inference for normal mixtures with unknown number of components
• ADAPTIVE GMM SHRINKAGE ESTIMATION WITH CONSISTENT MOMENT SELECTION
• Confidence sets based on penalized maximum likelihood estimators in Gaussian regression
• Distributional results for thresholding estimators in high-dimensional Gaussian regression models
• On the residual empirical process based on the ALASSO in high dimensions and its functional oracle property
• Asymptotic properties of the residual bootstrap for Lasso estimators
• Estimating heterogeneous graphical models for discrete data with an application to roll call voting
• Moderately clipped Lasso
• SHRINKAGE ESTIMATION OF REGRESSION MODELS WITH MULTIPLE STRUCTURAL CHANGES
• Comments on: \(\ell_{1}\)-penalization for mixture regression models
• Quantifying the cost of simultaneous non-parametric approximation of several samples
• Sparse estimation from noisy observations of an overdetermined linear system
• A Bayesian approach to sparse dynamic network identification
• Quantile-based portfolios: post-model-selection estimation with alternative specifications
• Finite-sample results for lasso and stepwise Neyman-orthogonal Poisson estimators
• The robust desparsified lasso and the focused information criterion for high-dimensional generalized linear models
• A robust test for network generated dependence
• Lasso Inference for High-Dimensional Time Series
• Title not available (Why is that?)
• Lassoing the Determinants of Retirement
• Simultaneous multiple non-crossing quantile regression estimation using kernel constraints
• Testing Sparsity-Inducing Penalties
• Valid post-selection inference
• Jump estimation in inverse regression
• Select the valid and relevant moments: an information-based Lasso for GMM with many moments
• The GENIUS approach to robust Mendelian randomization inference
• Valid Post-Selection Inference in High-Dimensional Approximately Sparse Quantile Regression Models
• Robust inference on average treatment effects with possibly more covariates than observations
• Exact adaptive confidence intervals for linear regression coefficients
• AUTOMATED ESTIMATION OF VECTOR ERROR CORRECTION MODELS
• On the distribution of penalized maximum likelihood estimators: the LASSO, SCAD, and thresholding
• \(\ell_1\)-regularization of high-dimensional time-series models with non-Gaussian and heteroskedastic errors
• Consistencies and rates of convergence of jump-penalized least squares estimators
• The Risk of James–Stein and Lasso Shrinkage
• ArCo: an artificial counterfactual approach for high-dimensional panel time-series data
• Covariate selection with group lasso and doubly robust estimation of causal effects

This page was built for publication: Sparse estimators and the oracle property, or the return of Hodges' estimator

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q290948)