Conditional predictive inference post model selection

DOI10.1214/08-AOS660zbMATH Open1173.62026arXiv0908.3615OpenAlexW2079776777MaRDI QIDQ834366FDOQ834366

Hannes Leeb

Publication date: 19 August 2009

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: We give a finite-sample analysis of predictive inference procedures after model selection in regression with random design. The analysis is focused on a statistically challenging scenario where the number of potentially important explanatory variables can be infinite, where no regularity conditions are imposed on unknown parameters, where the number of explanatory variables in a "good" model can be of the same order as sample size and where the number of candidate models can be of larger order than sample size. The performance of inference procedures is evaluated conditional on the training sample. Under weak conditions on only the number of candidate models and on their complexity, and uniformly over all data-generating processes under consideration, we show that a certain prediction interval is approximately valid and short with high probability in finite samples, in the sense that its actual coverage probability is close to the nominal one and in the sense that its length is close to the length of an infeasible interval that is constructed by actually knowing the "best" candidate model. Similar results are shown to hold for predictive inference procedures other than prediction intervals like, for example, tests of whether a future response will lie above or below a given threshold.

Full work available at URL: https://arxiv.org/abs/0908.3615

Recommendations

• Valid post-selection inference
• Valid confidence intervals for post-model-selection predictors
• Uniformly valid confidence intervals post-model-selection
• Exact post-selection inference, with application to the Lasso
• MODEL SELECTION AND INFERENCE: FACTS AND FICTION

zbMATH Keywords

approximately honest and short prediction intervalsconditional coverage probabilityfinite sample analysispredictive inference post model selectionregression with random design

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Linear regression; mixed models (62J05) Nonparametric tolerance and confidence regions (62G15) Estimation in multivariate analysis (62H12) Ridge regression; shrinkage estimators (Lasso) (62J07) Inequalities; stochastic orderings (60E15)

Cites Work

• Sparsity and Smoothness Via the Fused Lasso
• Honest confidence regions for nonparametric regression
• MODEL SELECTION AND INFERENCE: FACTS AND FICTION
• Prediction and asymptotics
• A Biometrics Invited Paper. The Analysis and Selection of Variables in Linear Regression
• Can one estimate the conditional distribution of post-model-selection estimators?
• Admissibility of the Usual Confidence Sets for the Mean of a Univariate or Bivariate Normal Population
• On the Large-Sample Minimal Coverage Probability of Confidence Intervals After Model Selection
• Evaluation and selection of models for out-of-sample prediction when the sample size is small relative to the complexity of the data-generating process
• CAN ONE ESTIMATE THE UNCONDITIONAL DISTRIBUTION OF POST-MODEL-SELECTION ESTIMATORS?
• THE FINITE-SAMPLE DISTRIBUTION OF POST-MODEL-SELECTION ESTIMATORS AND UNIFORM VERSUS NONUNIFORM APPROXIMATIONS
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Random rates in anisotropic regression. (With discussion)
• An adaptation theory for nonparametric confidence intervals
• Confidence sets for nonparametric wavelet regression
• Adaptive nonparametric confidence sets
• Modulation of estimators and confidence sets.
• Adaptive confidence bands
• Adaptive confidence balls
• How Many Variables Should be Entered in a Regression Equation?
• Confidence balls in Gaussian regression.
• The distribution of a linear predictor after model selection: conditional finite-sample distributions and asymptotic approximations
• Selection of Variables in Multiple Regression: Part II. Chosen Procedures, Computations and Examples
• Prediction Intervals, Factor Analysis Models, and High-Dimensional Empirical Linear Prediction
• Inference After Model Selection

Cited In (15)

• Forward-selected panel data approach for program evaluation
• Evaluation and selection of models for out-of-sample prediction when the sample size is small relative to the complexity of the data-generating process
• Inference after variable selection using restricted permutation methods
• Quasi-Bayesian model selection
• The relative effects of dimensionality and multiplicity of hypotheses on the \(F\)-test in linear regression
• Title not available (Why is that?)
• Post-model-selection prediction intervals for generalized linear models
• Optimal equivariant prediction for high-dimensional linear models with arbitrary predictor covariance
• Conditional conceptual predictive statistic for mixed model selection
• The coverage properties of confidence regions after model selection
• Model Selection Using Conditional Densities
• On asymptotic risk of selecting models for possibly nonstationary time-series
• Statistical Inference Enables Bad Science; Statistical Thinking Enables Good Science
• Can one estimate the conditional distribution of post-model-selection estimators?
• Ridge regression and asymptotic minimax estimation over spheres of growing dimension

This page was built for publication: Conditional predictive inference post model selection

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q834366)