Conditional predictive inference post model selection

DOI10.1214/08-AOS660MaRDI QIDQ834366zbMATH OpenOpenAlexFDO

Authors Hannes Leeb

Publication date 19 August 2009

Published in The Annals of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/0908.3615

zbMATH Keywords

approximately honest and short prediction intervalsconditional coverage probabilityfinite sample analysispredictive inference post model selectionregression with random design

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Linear regression; mixed models (62J05) Nonparametric tolerance and confidence regions (62G15) Estimation in multivariate analysis (62H12) Ridge regression; shrinkage estimators (Lasso) (62J07) Inequalities; stochastic orderings (60E15)

Abstract: We give a finite-sample analysis of predictive inference procedures after model selection in regression with random design. The analysis is focused on a statistically challenging scenario where the number of potentially important explanatory variables can be infinite, where no regularity conditions are imposed on unknown parameters, where the number of explanatory variables in a "good" model can be of the same order as sample size and where the number of candidate models can be of larger order than sample size. The performance of inference procedures is evaluated conditional on the training sample. Under weak conditions on only the number of candidate models and on their complexity, and uniformly over all data-generating processes under consideration, we show that a certain prediction interval is approximately valid and short with high probability in finite samples, in the sense that its actual coverage probability is close to the nominal one and in the sense that its length is close to the length of an infeasible interval that is constructed by actually knowing the "best" candidate model. Similar results are shown to hold for predictive inference procedures other than prediction intervals like, for example, tests of whether a future response will lie above or below a given threshold.

Recommendations

• Valid post-selection inference
• Valid confidence intervals for post-model-selection predictors
• Uniformly valid confidence intervals post-model-selection
• Exact post-selection inference, with application to the Lasso
• MODEL SELECTION AND INFERENCE: FACTS AND FICTION

Cites work

• scientific article; zbMATH DE number 3856278 (Why is no real title available?)
• scientific article; zbMATH DE number 735230 (Why is no real title available?)
• A Biometrics Invited Paper. The Analysis and Selection of Variables in Linear Regression
• Adaptive confidence balls
• Adaptive confidence bands
• Adaptive nonparametric confidence sets
• Admissibility of the Usual Confidence Sets for the Mean of a Univariate or Bivariate Normal Population
• An adaptation theory for nonparametric confidence intervals
• CAN ONE ESTIMATE THE UNCONDITIONAL DISTRIBUTION OF POST-MODEL-SELECTION ESTIMATORS?
• Can one estimate the conditional distribution of post-model-selection estimators?
• Confidence balls in Gaussian regression.
• Confidence sets for nonparametric wavelet regression
• Evaluation and selection of models for out-of-sample prediction when the sample size is small relative to the complexity of the data-generating process
• Honest confidence regions for nonparametric regression
• How Many Variables Should be Entered in a Regression Equation?
• Inference After Model Selection
• MODEL SELECTION AND INFERENCE: FACTS AND FICTION
• Modulation of estimators and confidence sets.
• On the Large-Sample Minimal Coverage Probability of Confidence Intervals After Model Selection
• Prediction Intervals, Factor Analysis Models, and High-Dimensional Empirical Linear Prediction
• Prediction and asymptotics
• Random rates in anisotropic regression. (With discussion)
• Selection of Variables in Multiple Regression: Part II. Chosen Procedures, Computations and Examples
• Sparsity and Smoothness Via the Fused Lasso
• THE FINITE-SAMPLE DISTRIBUTION OF POST-MODEL-SELECTION ESTIMATORS AND UNIFORM VERSUS NONUNIFORM APPROXIMATIONS
• The distribution of a linear predictor after model selection: conditional finite-sample distributions and asymptotic approximations
• The distribution of a linear predictor after model selection: unconditional finite-sample distributions and asymptotic approximations

Cited in

(17)

• Inference after variable selection using restricted permutation methods
• Post-model-selection prediction intervals for generalized linear models
• Valid post-selection inference
• Optimal equivariant prediction for high-dimensional linear models with arbitrary predictor covariance
• Can one estimate the conditional distribution of post-model-selection estimators?
• Model Selection Using Conditional Densities
• Quasi-Bayesian model selection
• Forward-selected panel data approach for program evaluation
• The relative effects of dimensionality and multiplicity of hypotheses on the \(F\)-test in linear regression
• Evaluation and selection of models for out-of-sample prediction when the sample size is small relative to the complexity of the data-generating process
• On asymptotic risk of selecting models for possibly nonstationary time-series
• Statistical Inference Enables Bad Science; Statistical Thinking Enables Good Science
• Ridge regression and asymptotic minimax estimation over spheres of growing dimension
• Empirical priors for prediction in sparse high-dimensional linear regression
• The coverage properties of confidence regions after model selection
• Conditional predictive inference for stable algorithms
• Conditional conceptual predictive statistic for mixed model selection

This page was built for publication: Conditional predictive inference post model selection

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q834366)