Valid post-selection inference

DOI10.1214/12-AOS1077zbMATH Open1267.62080arXiv1306.1059OpenAlexW2009462809MaRDI QIDQ355109FDOQ355109

Authors: Andreas Buja, Linda Zhao, Richard Berk, Lawrence Brown, Kai Zhang 

Publication date: 24 July 2013

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: It is common practice in statistical data analysis to perform data-driven variable selection and derive statistical inference from the resulting model. Such inference enjoys none of the guarantees that classical statistical theory provides for tests and confidence intervals when the model has been chosen a priori. We propose to produce valid ``post-selection inference by reducing the problem to one of simultaneous inference and hence suitably widening conventional confidence and retention intervals. Simultaneity is required for all linear functions that arise as coefficient estimates in all submodels. By purchasing ``simultaneity insurance for all possible submodels, the resulting post-selection inference is rendered universally valid under all possible model selection procedures. This inference is therefore generally conservative for particular selection procedures, but it is always less conservative than full Scheffe protection. Importantly it does not depend on the truth of the selected submodel, and hence it produces valid inference even in wrong models. We describe the structure of the simultaneous inference problem and give some asymptotic results.

Full work available at URL: https://arxiv.org/abs/1306.1059

Recommendations

• Exact post-selection inference, with application to the Lasso
• Valid post-selection inference in model-free linear regression
• MODEL SELECTION AND INFERENCE: FACTS AND FICTION
• Selective inference after likelihood- or test-based model selection in linear models
• Conditional predictive inference post model selection

zbMATH Keywords

model selectionhigh-dimensional inferencelinear regressionmultiple comparisonssphere packingfamily-wise error

Mathematics Subject Classification ID

Linear regression; mixed models (62J05) Estimation in multivariate analysis (62H12) Analysis of variance and covariance (ANOVA) (62J10) Paired and multiple comparisons; multiple testing (62J15)

Cites Work

• Mostly harmless econometrics. An empiricist's companion.
• A Note on Quantiles in Large Samples
• Sparse estimators and the oracle property, or the return of Hodges' estimator
• Title not available (Why is that?)
• Valid post-selection inference
• MODEL SELECTION AND INFERENCE: FACTS AND FICTION
• On the distribution of penalized maximum likelihood estimators: the LASSO, SCAD, and thresholding
• Can one estimate the conditional distribution of post-model-selection estimators?
• Frequentist Model Average Estimators
• The Focused Information Criterion
• On the Large-Sample Minimal Coverage Probability of Confidence Intervals After Model Selection
• On the distribution of the adaptive LASSO estimator
• On preliminary test and shrinkage M-estimation in linear models
• Confidence sets based on penalized maximum likelihood estimators in Gaussian regression
• Distributional results for thresholding estimators in high-dimensional Gaussian regression models
• The Conditional Level of the F-Test
• PERFORMANCE LIMITS FOR ESTIMATORS OF THE RISK OR DISTRIBUTION OF SHRINKAGE-TYPE ESTIMATORS, AND SOME GENERAL LOWER RISK-BOUND RESULTS
• CAN ONE ESTIMATE THE UNCONDITIONAL DISTRIBUTION OF POST-MODEL-SELECTION ESTIMATORS?
• THE FINITE-SAMPLE DISTRIBUTION OF POST-MODEL-SELECTION ESTIMATORS AND UNIFORM VERSUS NONUNIFORM APPROXIMATIONS
• The distribution of a linear predictor after model selection: unconditional finite-sample distributions and asymptotic approximations
• The distribution of model averaging estimators and an impossibility result regarding its estima\-tion
• The Conditional Level of Student's $t$ Test
• Note on a Conditional Property of Student's $t^1$
• Asymptotic properties of maximum likelihood estimators based on conditional specification
• Title not available (Why is that?)
• Title not available (Why is that?)
• On model uncertainty and its statistical implications. Proceedings of a workshop, held in Groningen, Netherlands, September 25-26, 1986
• Random Packings and Coverings of the Unit n-Sphere

Cited In (only showing first 100 items - show all)

• False Discovery Rate Control via Data Splitting
• Forward-selected panel data approach for program evaluation
• Selection of mixed copula for association modeling with tied observations
• Exploration of the variability of variable selection based on distances between bootstrap sample results
• FANOK: knockoffs in linear time
• On Hodges' superefficiency and merits of oracle property in model selection
• The costs and benefits of uniformly valid causal inference with high-dimensional nuisance parameters
• Penalized likelihood and multiple testing
• Post hoc confidence bounds on false positives using reference families
• Inferactive data analysis
• Title not available (Why is that?)
• Optimal configurations of lines and a statistical application
• Targeted Inference Involving High-Dimensional Data Using Nuisance Penalized Regression
• On the length of post-model-selection confidence intervals conditional on polyhedral constraints
• Post-model-selection inference in linear regression models: an integrated review
• On the impact of model selection on predictor identification and parameter inference
• On the least-squares model averaging interval estimator
• Post-selection inference via algorithmic stability
• A multi-resolution theory for approximating infinite-\(p\)-zero-\(n\): transitional inference, individualized predictions, and a world without bias-variance tradeoff
• Only closed testing procedures are admissible for controlling false discovery proportions
• Multicarving for high-dimensional post-selection inference
• Score Tests With Incomplete Covariates and High-Dimensional Auxiliary Variables
• Excess optimism: how biased is the apparent error of an estimator tuned by SURE?
• Bayesian semiparametric functional mixed models for serially correlated functional data, with application to glaucoma data
• Log-linear Bayesian additive regression trees for multinomial logistic and count regression models
• In defense of the indefensible: a very naïve approach to high-dimensional inference
• Regularized projection score estimation of treatment effects in high-dimensional quantile regression
• Constraints versus priors
• Optimal finite sample post-selection confidence distributions in generalized linear models
• A nonparametric sequential learning procedure for estimating the pure premium
• Bayesian Inference Is Unaffected by Selection: Fact or Fiction?
• Assumption Lean Regression
• Selective inference after feature selection via multiscale bootstrap
• Title not available (Why is that?)
• Rejoinder on: ``Hierarchical inference for genome-wide association studies: a view on methodology with software
• Informative goodness-of-fit for multivariate distributions
• Confidence intervals for parameters in high-dimensional sparse vector autoregression
• Some perspectives on inference in high dimensions
• Principled statistical inference in data science
• Models as approximations. I. Consequences illustrated with linear regression
• Neighborhood-based cross fitting approach to treatment effects with high-dimensional data
• Spatially relaxed inference on high-dimensional linear models
• Inference for \(L_2\)-boosting
• Larry Brown's contributions to parametric inference, decision theory and foundations: a survey
• Statistical theory powering data science
• Conditional selective inference for robust regression and outlier detection using piecewise-linear homotopy continuation
• Robust Q-learning
• Selection-corrected statistical inference for region detection with high-throughput assays
• Statistical proof? The problem of irreproducibility
• Frequentist model averaging in structural equation modelling
• Post-selection inference of generalized linear models based on the lasso and the elastic net
• Confidently Comparing Estimates with the c-value
• Distributionally robust and generalizable inference
• Markov Neighborhood Regression for High-Dimensional Inference
• An evolutionary estimation procedure for generalized semilinear regression trees
• Penalized estimation of a class of single‐index varying‐coefficient models for integrative genomic analysis
• Exact post-selection inference for adjusted R squared selection
• Optimal model averaging for divergent-dimensional Poisson regressions
• The Perils of Balance Testing in Experimental Design: Messy Analyses of Clean Data
• Ensuring valid inference for Cox hazard ratios after variable selection
• Sparse estimation in semiparametric finite mixture of varying coefficient regression models
• Variable Selection for Global Fréchet Regression
• High-dimensional statistical inference via DATE
• Trade-off between predictive performance and FDR control for high-dimensional Gaussian model selection
• Projection-based techniques for high-dimensional optimal transport problems
• Integrative analysis of `-omics' data using penalty functions
• Locally simultaneous inference
• Integrative Bayesian Models Using Post-Selective Inference: A Case Study in Radiogenomics
• Empirical likelihood based tests for detecting the presence of significant predictors in marginal quantile regression
• Cellwise outlier detection with false discovery rate control
• A structured brain‐wide and genome‐wide association study using ADNI PET images
• Post-selection estimation and testing following aggregate association tests
• Carving model-free inference
• Approximate Selective Inference via Maximum Likelihood
• Unlucky Number 13? Manipulating Evidence Subject to Snooping
• Markov neighborhood regression for statistical inference of high-dimensional generalized linear models
• Conformal Prediction Credibility Intervals
• A comparison of full model specification and backward elimination of potential confounders when estimating marginal and conditional causal effects on binary outcomes from observational data
• Asymptotically Uniform Tests After Consistent Model Selection in the Linear Regression Model
• Efficient interaction selection for clustered data via stagewise generalized estimating equations
• Confounder selection strategies targeting stable treatment effect estimators
• Comments on “Unobservable Selection and Coefficient Stability: Theory and Evidence” and “Poorly Measured Confounders are More Useful on the Left Than on the Right”
• Controlling False Discovery Rate Using Gaussian Mirrors
• Post-model-selection prediction intervals for generalized linear models
• Discovery and Inference of a Causal Network with Hidden Confounding
• Prediction intervals with controlled length in the heteroscedastic Gaussian regression
• A Normality Test for High-dimensional Data Based on the Nearest Neighbor Approach
• Filtering the Rejection Set While Preserving False Discovery Rate Control
• Scalable and efficient inference via CPE
• Statistical Inference Enables Bad Science; Statistical Thinking Enables Good Science
• A (tight) upper bound for the length of confidence intervals with conditional coverage
• Forward stability and model path selection
• Heterogeneous heterogeneity by default: Testing categorical moderators in mixed‐effects meta‐analysis
• SLOPE-adaptive variable selection via convex optimization
• Inference for low‐ and high‐dimensional inhomogeneous Gibbs point processes
• High-dimensional inference: confidence intervals, \(p\)-values and R-software \texttt{hdi}
• On various confidence intervals post-model-selection
• Inference After Model Selection
• Selective inference for latent block models
• Simultaneous high-probability bounds on the false discovery proportion in structured, regression and online settings

Uses Software

• ElemStatLearn

This page was built for publication: Valid post-selection inference

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q355109)