Estimating the Error Rate of a Prediction Rule: Improvement on Cross-Validation
This paper addresses the following problem: a prediction rule (model) is constructed on the basis of some data and then it is desired to estimate the error (err) of this rule in predicting future observations. The cross-validation, bootstrap, randomized bootstrap, and double bootstrap estimators of err are discussed in detail and analysed. The relations between these estimators are also established. The paper is mainly concerned with the situation when the observations and their predictions are dichotomous and the input variables are independent and identically distributed.
- Improvements on Cross-Validation: The .632+ Bootstrap Method
- A comparison of nonparametric error rate estimation methods in classification problems
- Assessing the performance of an allocation rule
- Measuring the prediction error. A comparison of cross-validation, bootstrap and covariance penalty methods
- Estimating classification error rate: repeated cross-validation, repeated hold-out and bootstrap
- Fast and robust bootstrap
- Bootstrap variants of the Akaike information criterion for mixed model selection
- Asymptotic bootstrap corrections of AIC for linear regression models
- Re-sampling procedures for reducing bias of error rate estimation in multinomial classification
- A shrunken leaving-one-out estimator of error rate
- Large sample properties of cross-validation assessment statistics
- Estimating recorder points and other management science applications by bootstrap procedure
- On model selection via stochastic complexity in robust linear regression
- A piecewise transition model for analyzing multistate life history data
- A bootstrap generalized likelihood ratio test in discriminant analysis
- Bootstrap by sequential resampling
- Model selection criteria based on cross-validatory concordance statistics
- Extending AIC to best subset regression
- Order-based error for managing ensembles of surrogates in mesh adaptive direct search
- Bolstered error estimation
- Bandwidth choice for nonparametric classification
- Assessing classifiers in terms of the partial area under the ROC curve
- The use of a distance measure in regularised discriminant analysis
- Evaluating the impact of exploratory procedures in regression prediction: A pseudosample approach
- On the biases of error estimators in prediction problems
- Surrogates for numerical simulations; optimization of eddy-promoter heat exchangers
- Resampling-based information criteria for best-subset regression
- Optimal model selection in heteroscedastic regression using piecewise polynomial functions
- Model selection by resampling penalization
- Sparse supervised dimension reduction in high dimensional classification
- Asymptotic optimality of full cross-validation for selecting linear regression models
- A large-sample model selection criterion based on Kullback's symmetric divergence
- Unsupervised stratification of cross-validation for accuracy estimation
- A gamma kernel density estimation for insurance loss data
- Comment: Automated analyses: because we can, does it mean we should?
- ROS regression: integrating regularization with optimal scaling regression
- Validation in principal components analysis applied to EEG data
- On the predictive risk in misspecified quantile regression
- Comparing and selecting spatial predictors using local criteria
- Cross-conformal predictors
- A new variable selection approach using random forests
- Model-free model-fitting and predictive distributions
- Measuring the prediction error. A comparison of cross-validation, bootstrap and covariance penalty methods
- Fast robust estimation of prediction error based on resampling
- Model combination for credit risk assessment: a stacked generalization approach
- Multi-class tumor classification by discriminant partial least squares using microarray gene expression data and assessment of classification models
- Optimal convex error estimators for classification
- Bayesian nonparametric model selection and model testing
- Model selection by normalized maximum likelihood
- Exact performance of error estimators for discrete classifiers
- Greedy algorithms for prediction
- Tree-structured prognostic classification for censored survival data: validation of computationally inexpensive model selection criteria
- A comparison of parametric conditional error-rate estimators for the two-group linear discriminant function
- Assessing incremental value of biomarkers with multi-phase nested case-control studies
- Estimating prediction error in microarray classification: modifications of the 0.632+ bootstrap when \(n<p\)
- Iterative bias correction of the cross-validation criterion
- Optimal combinations of diagnostic tests based on AUC
- Post-hoc analyses in multiple regression based on prediction error
- Integration of rough set and neural network ensemble to predict the configuration performance of a modular product family
- Multiple group linear discriminant analysis: robustness and error rate
- Cross-Validation, Risk Estimation, and Model Selection: Comment on a Paper by Rosset and Tibshirani
- A method for constructing a confidence bound for the actual error rate of a prediction rule in high dimensions
- Combining neural networks for function approximation under conditions of sparse data: the biased regression approach
- On stability issues in deriving multivariable regression models
- A serial risk score approach to disease classification that accounts for accuracy and cost
- The use of Smooth Bootstrap Techniques for Estimating the Error Rate of a Prediction Rule
- On error-rate estimation in nonparametric classification
- Estimating prediction error: cross-validation vs. accumulated prediction error
- Maximizing proportions of correct classifications in binary logistic regression
- Improved model identification for non-linear systems using a random subsampling and multifold modelling (RSMM) approach
- Empirical comparison of error rate-estimators in logistic discriminant analysis
- New Bootstrap Applications in Supervised Learning
- Assessing the performance of an allocation rule
- The robust estimation of classification error rates
- How Biased is the Apparent Error Rate of a Prediction Rule?
- A note on estimating the msep in nonlinear regression
- DATA-DEPENDENT ESTIMATION OF PREDICTION FUNCTIONS
- Optimal mean-square-error calibration of classifier error estimators under Bayesian models
- Selection of regressors in econometrics: parametric and nonparametric methods selection of regressors in econometrics
- Validation of average error rate over classifiers
- A theory of cross-validation error
- Determination of the best significance level in forward stepwise logistic regression
- A conversation with Seymour Geisser
- Margin-adaptive model selection in statistical learning
- Data-based interval estimation of classification error rates
- The out-of-source error in multi-source cross validation-type procedures
- Evaluating Decision Rules for Nitrogen Fertilization
- A Modified Akaike Criterion for Model Choice in Generalized Linear Models
- Ten More Years of Error Rate Research
- On the estimation of prediction errors in logistic regression models
- Discriminant analysis with stratified prior probabilities
- Estimationg error rates in discriminant analysis with correlated training observations: a simulation study
- Provably safe and robust learning-based model predictive control
- Cross‐validation and peeling strategies for survival bump hunting using recursive peeling methods
- Improved Cross-Validation for Classifiers that Make Algorithmic Choices to Minimise Runtime Without Compromising Output Correctness
- A flexible and efficient spatial interpolator for radar rainfall estimation
- The comparison study of the model selection criteria on the Tobit regression model based on the bootstrap sample augmentation mechanisms
- Comparison of classical tumour growth models for patient derived and cell-line derived xenografts using the nonlinear mixed-effects framework
- Model selection in linear regression using paired bootstrap
- Bootstrap estimation and model selection for multivariate normal mixtures using parallel computing with graphics processing units
- Autoregressive model selection based on a prediction perspective
- Sequential regression measurement error models with application
- Improved small-sample estimation of nonlinear cross-validated prediction metrics
- Adaptive order selection for autoregressive models
- Degrees of freedom in low rank matrix estimation
This page was built for publication: Estimating the Error Rate of a Prediction Rule: Improvement on Cross-Validation
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3332119)