Evaluation and selection of models for out-of-sample prediction when the sample size is small relative to the complexity of the data-generating process

DOI10.3150/08-BEJ127MaRDI QIDQ1002545zbMATH OpenOpenAlexFDO

Authors Hannes Leeb

Publication date 2 March 2009

Published in Bernoulli (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/0802.3364

zbMATH Keywords

\(S_p\) criteriongeneralized cross validationlarge number of parameters and small sample sizemodel selectionnonparametric regressionout-of-sample prediction

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Asymptotic properties of nonparametric inference (62G20)

Abstract: In regression with random design, we study the problem of selecting a model that performs well for out-of-sample prediction. We do not assume that any of the candidate models under consideration are correct. Our analysis is based on explicit finite-sample results. Our main findings differ from those of other analyses that are based on traditional large-sample limit approximations because we consider a situation where the sample size is small relative to the complexity of the data-generating process, in the sense that the number of parameters in a `good' model is of the same order as sample size. Also, we allow for the case where the number of candidate models is (much) larger than sample size.

Recommendations

• Model selection for small sample regression
• Conditional predictive inference post model selection
• On model selection in the computer age
• scientific article; zbMATH DE number 5769866
• Study of a simple method and its consistency for selecting regression models in fixed and random designs

Cites work

• A Biometrics Invited Paper. The Analysis and Selection of Variables in Linear Regression
• A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations
• Admissible variable-selection procedures when fitting regression models by least squares for prediction
• An Elementary Theorem on the Probability of Large Deviations
• Asymptotic behavior of M estimators of p regression parameters when \(p^ 2/n\) is large. II: Normal approximation
• Asymptotic behavior of M-estimators of p regression parameters when p^ 2/n is large. I. Consistency
• Asymptotics with increasing dimension for robust regression with applications to the bootstrap
• Can the strengths of AIC and BIC be shared? A conflict between model indentification and regression estimation
• Conditional predictive inference post model selection
• Confidence sets centered at \(C_ p\)-estimators
• DISTRIBUTION OF EIGENVALUES FOR SOME SETS OF RANDOM MATRICES
• Estimating the dimension of a model
• Fitting autoregressive models for prediction
• How Many Variables Should be Entered in a Regression Equation?
• scientific article; zbMATH DE number 1321826 (Why is no real title available?)
• scientific article; zbMATH DE number 1034037 (Why is no real title available?)
• scientific article; zbMATH DE number 3291685 (Why is no real title available?)
• MODEL SELECTION AND INFERENCE: FACTS AND FICTION
• Model selection for regression on a random design
• Model selection in nonparametric regression
• Modeling by shortest data description
• Modulation of estimators and confidence sets.
• Optimal prediction for linear regression with infinitely many parameters.
• REACT Scatterplot Smoothers: Superefficiency through Basis Economy
• Regression and time series model selection in small samples
• Risk bounds for model selection via penalization
• Robust regression: Asymptotics, conjectures and Monte Carlo
• Selection of Regressors
• Selection of Variables in Multiple Regression: Part I. A Review and Evaluation
• Selection of Variables in Multiple Regression: Part II. Chosen Procedures, Computations and Examples
• Smoothing noisy data with spline functions: Estimating the correct degree of smoothing by the method of generalized cross-validation
• Some Comments on C P
• Sparse estimators and the oracle property, or the return of Hodges' estimator
• Statistical predictor identification
• The Focused Information Criterion
• The risk inflation criterion for multiple regression

Cited in

(11)

• Generalized cross validation in variable selection with and without shrinkage
• On various confidence intervals post-model-selection
• scientific article; zbMATH DE number 5808216 (Why is no real title available?)
• From Fixed-X to Random-X Regression: Bias-Variance Decompositions, Covariance Penalties, and Prediction Error Estimation
• Computer Experiments: Prediction Accuracy, Sample Size and Model Complexity Revisited
• Model selection for small sample regression
• When are Google Data Useful to Nowcast GDP? An Approach via Preselection and Shrinkage
• Parametric or nonparametric? A parametricness index for model selection
• Information criteria for model selection
• Corrected generalized cross-validation for finite ensembles of penalized estimators
• Conditional predictive inference post model selection

This page was built for publication: Evaluation and selection of models for out-of-sample prediction when the sample size is small relative to the complexity of the data-generating process

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1002545)