Consistency of cross validation for comparing regression procedures
From MaRDI portal
Publication:2473071
Abstract: Theoretical developments on cross validation (CV) have mainly focused on selecting one among a list of finite-dimensional models (e.g., subset or order selection in linear regression) or selecting a smoothing parameter (e.g., bandwidth for kernel smoothing). However, little is known about consistency of cross validation when applied to compare between parametric and nonparametric methods or within nonparametric methods. We show that under some conditions, with an appropriate choice of data splitting ratio, cross validation is consistent in the sense of selecting the better procedure with probability approaching 1. Our results reveal interesting behavior of cross validation. When comparing two models (procedures) converging at the same nonparametric rate, in contrast to the parametric case, it turns out that the proportion of data used for evaluation in CV does not need to be dominating in size. Furthermore, it can even be of a smaller order than the proportion for estimation while not affecting the consistency property.
Recommendations
- Cross-validation for comparing multiple density estimation procedures
- scientific article; zbMATH DE number 5056254
- Cross-validation for selecting a model selection procedure
- A survey of cross-validation procedures for model selection
- On the consistency of cross-validation in kernel nonparametric regression
Cites work
- scientific article; zbMATH DE number 991833 (Why is no real title available?)
- scientific article; zbMATH DE number 3860199 (Why is no real title available?)
- scientific article; zbMATH DE number 20176 (Why is no real title available?)
- scientific article; zbMATH DE number 45848 (Why is no real title available?)
- scientific article; zbMATH DE number 3483405 (Why is no real title available?)
- scientific article; zbMATH DE number 410127 (Why is no real title available?)
- scientific article; zbMATH DE number 1034037 (Why is no real title available?)
- scientific article; zbMATH DE number 2015216 (Why is no real title available?)
- scientific article; zbMATH DE number 1522808 (Why is no real title available?)
- scientific article; zbMATH DE number 3444596 (Why is no real title available?)
- A comparative study of ordinary cross-validation, v-fold cross-validation and the repeated learning-testing methods
- A distribution-free theory of nonparametric regression
- Adaptive Regression by Mixing
- Applied Linear Regression
- Asymptotic optimality for \(C_ p\), \(C_ L\), cross-validation and generalized cross-validation: Discrete index set
- Consistency for cross-validated nearest neighbor estimates in nonparametric regression
- Convergence of stochastic processes
- How Far Are Automatically Chosen Regression Smoothing Parameters From Their Optimum?
- Linear Model Selection by Cross-Validation
- Minimax estimation via wavelet shrinkage
- Model selection in nonparametric regression
- Model selection via multifold cross validation
- Nonparametric regression with correlated errors.
- Nonparametric smoothing and lack-of-fit tests
- On the consistency of cross-validation in kernel nonparametric regression
- Optimal global rates of convergence for nonparametric regression
- Optimal rates of convergence for nonparametric estimators
- Oracle inequalities for multi-fold cross validation
- Smoothing methods in statistics
- Smoothing noisy data with spline functions: Estimating the correct degree of smoothing by the method of generalized cross-validation
- Spline smoothing and optimal rates of convergence in nonparametric regression models
- The Predictive Sample Reuse Method with Applications
- The Relationship between Variable Selection and Data Agumentation and a Method for Prediction
- The cross-validated adaptive epsilon-net estimator
Cited in
(41)- Sparsity oriented importance learning for high-dimensional linear regression
- A cross-validation based estimation of the proportion of true null hypotheses
- A survey of Bayesian predictive methods for model assessment, selection and comparison
- Parametric or nonparametric? A parametricness index for model selection
- Double-slicing assisted sufficient dimension reduction for high-dimensional censored data
- Cross-Validation: What Does It Estimate and How Well Does It Do It?
- Estimation of prediction error by using \(K\)-fold cross-validation
- Targeted cross-validation
- Model selection via standard error adjusted adaptive Lasso
- Catching up Faster by Switching Sooner: A Predictive Approach to Adaptive Estimation with an Application to the AIC–BIC Dilemma
- Model selection by resampling penalization
- Estimating the Kullback–Liebler risk based on multifold cross‐validation
- Cross-validation with confidence
- Penalized cluster analysis with applications to family data
- Cross-validation for change-point regression: pitfalls and solutions
- Cross-validation for comparing multiple density estimation procedures
- Multiple predicting \(K\)-fold cross-validation for model selection
- Consistency of empirical Bayes and kernel flow for hierarchical parameter estimation
- Determining the number of factors in approximate factor models by twice K-fold cross validation
- Estimating and forecasting dynamic correlation matrices: a nonlinear common factor approach
- Efficient, adaptive cross-validation for tuning and comparing models, with application to drug discovery
- On consistent statistical procedures in regression
- A Note on Cross-Validation for Lasso Under Measurement Errors
- Mixing partially linear regression models
- Consistent selection of the number of change-points via sample-splitting
- Degrees of freedom in submodular regularization: a computational perspective of Stein's unbiased risk estimate
- Robustness by reweighting for kernel estimators: an overview
- Regression in Tensor Product Spaces by the Method of Sieves
- Segmentation of the mean of heteroscedastic data via cross-validation
- A survey of cross-validation procedures for model selection
- Cross-Validation, Risk Estimation, and Model Selection: Comment on a Paper by Rosset and Tibshirani
- Theoretical analysis of cross-validation for estimating the risk of the \(k\)-nearest neighbor classifier
- Bayes shrinkage estimation for high-dimensional VAR models with scale mixture of normal distributions for noise
- Performance Assessment of High-dimensional Variable Identification
- Variable selection in convex quantile regression: \(\mathcal{L}_1\)-norm or \(\mathcal{L}_0\)-norm regularization?
- Asymptotics of K-fold cross validation
- Risk consistency of cross-validation with Lasso-type procedures
- Cross-validation for selecting a model selection procedure
- Equivalence of regression calibration methods in main study/external validation study designs
- Consistent estimation of the number of communities in stochastic block models using cross-validation
- The art of transfer learning: an adaptive and robust pipeline
This page was built for publication: Consistency of cross validation for comparing regression procedures
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2473071)
