How Biased is the Apparent Error Rate of a Prediction Rule?

DOI10.2307/2289236MaRDI QIDQ3757198zbMATH OpenOpenAlexFDO

Authors Bradley Efron

Publication date 1986

Full work available at URL https://doi.org/10.2307/2289236

zbMATH Keywords

bootstrapgeneralized linear modelAICcross validationexamplelogistic modelprediction modeloptimismNumerical resultsunderestimationprediction errorsdownward biasMallow's CpAkaike criteriumerror rate of prediction rulegeneral exponential family linear modelsmeasures of prediction errors

Mathematics Subject Classification ID

Point estimation (62F10) Linear regression; mixed models (62J05) Linear inference, regression (62J99) Parametric inference (62F99)

Recommendations

• Estimating the Error Rate of a Prediction Rule: Improvement on Cross-Validation
• On the estimation of prediction errors in logistic regression models
• Overestimation of the receiver operating characteristic curve for logistic regression
• On the biases of error estimators in prediction problems
• Assessing the performance of an allocation rule

Cited in

(only showing first 100 items - show all)

• Compressed covariance estimation with automated dimension learning
• A regression model selection criterion based on bootstrap bumping for use with resistant fitting.
• Discussion: ``A significance test for the lasso
• A survey of cross-validation procedures for model selection
• Determination of the best significance level in forward stepwise logistic regression
• Asymptotic optimality of full cross-validation for selecting linear regression models
• Bayesian nonparametric model selection and model testing
• Reluctant generalized additive modeling
• Flexible and Interpretable Models for Survival Data
• High-Dimensional Spatial Quantile Function-on-Scalar Regression
• Optimality of training/test size and resampling effectiveness in cross-validation
• Variable selection for generalized linear mixed models by \(L_1\)-penalized estimation
• Prediction Error Estimation Under Bregman Divergence for Non‐Parametric Regression and Classification
• Prediction-based structured variable selection through the receiver operating characteristic curves
• Bayesian comparison of latent variable models: conditional versus marginal likelihoods
• A model search procedure for hierarchical models
• From Fixed-X to Random-X Regression: Bias-Variance Decompositions, Covariance Penalties, and Prediction Error Estimation
• Extreme value correction: a method for correcting optimistic estimations in rule learning
• Honest leave-one-out cross-validation for estimating post-tuning generalization error
• scientific article; zbMATH DE number 7307469 (Why is no real title available?)
• Degrees of freedom and model selection for \(k\)-means clustering
• Bootstrap-based model selection criteria for beta regressions
• Degrees of freedom in low rank matrix estimation
• Statistical significance of the Netflix challenge
• Efficient Computation and Model Selection for the Support Vector Regression
• Recent developments in bootstrap methodology
• Least angle regression. (With discussion)
• Bootstrap estimation and model selection for multivariate normal mixtures using parallel computing with graphics processing units
• A study on tuning parameter selection for the high-dimensional lasso
• Multiple group linear discriminant analysis: robustness and error rate
• Assessing the performance of data assimilation algorithms which employ linear error feedback
• Sparse estimation via nonconcave penalized likelihood in factor analysis model
• On the optimism correction of the area under the receiver operating characteristic curve in logistic prediction models
• A Pliable Lasso
• On the estimation of prediction errors in logistic regression models
• A non-convex regularization approach for stable estimation of loss development factors
• Cross-Validation: What Does It Estimate and How Well Does It Do It?
• Distance-based linear discriminant analysis for interval-valued data
• Discussion: ``A significance test for the lasso
• Discussion: ``A significance test for the lasso
• Discussion: ``A significance test for the lasso
• Discussion: ``A significance test for the lasso
• New aspects of Bregman divergence in regression and classification with parametric and nonparametric estimation
• Are ordinal models useful for classification? a revised analysis
• Smoothing spline ANOVA models for large data sets with Bernoulli observations and the randomized GACV.
• Local behavior of sparse analysis regularization: applications to risk estimation
• P-splines with an \(\ell_1\) penalty for repeated measures
• Using specially designed exponential families for density estimation
• The asymptotic distribution of the proportion of correct classifications for a holdout sample in logistic regression
• Comparing and selecting spatial predictors using local criteria
• Fused Lasso nearly-isotonic signal approximation in general dimensions
• Estimation of the conditional risk in classification: the swapping method
• A significance test for the lasso
• Bootstrap variants of the Akaike information criterion for mixed model selection
• On the biases of error estimators in prediction problems
• Maximizing proportions of correct classifications in binary logistic regression
• Reconceptualizing the p -value from a likelihood ratio test: a probabilistic pairwise comparison of models based on Kullback-Leibler discrepancy measures
• Inference after variable selection using restricted permutation methods
• Model selection by resampling penalization
• Cross validation model selection criteria for linear regression based on the Kullback-Leibler discrepancy
• The degrees of freedom of partly smooth regularizers
• Modeling strategies in longitudinal data analysis: covariate, variance function and correlation structure selection
• Adapting to unknown sparsity by controlling the false discovery rate
• Is \(C_{p}\) an empirical Bayes method for smoothing parameter choice?
• On model selection via stochastic complexity in robust linear regression
• Discussion: ``A significance test for the lasso
• Low complexity regularization of linear inverse problems
• Asymptotic bootstrap corrections of AIC for linear regression models
• Asymptotic properties of a double penalized maximum likelihood estimator in logistic regres\-sion
• Evaluation of generalized degrees of freedom for sparse estimation by replica method
• Estimating the Kullback–Liebler risk based on multifold cross‐validation
• Evaluating the impact of exploratory procedures in regression prediction: A pseudosample approach
• Optimal Simulator Selection
• Additive models with trend filtering
• Ideal point discriminant analysis
• Modelling of insurers' rating determinants. An application of machine learning techniques and statistical models
• Discussion of “From Fixed-X to Random-X Regression: Bias-Variance Decompositions, Covariance Penalties, and Prediction Error Estimation”
• Estimating the accuracy of (local) cross-validation via randomised GCV choices in kernel or smoothing spline regression
• Variable selection in canonical discriminant analysis for family studies
• Degrees of freedom for off-the-grid sparse estimation
• Tuning parameter selection in sparse regression modeling
• Measuring the prediction error. A comparison of cross-validation, bootstrap and covariance penalty methods
• Criterion constrained Bayesian hierarchical models
• Regular, median and Huber cross‐validation: A computational comparison
• Cross-Validation for Correlated Data
• Extending AIC to best subset regression
• Model selection criteria based on cross-validatory concordance statistics
• Selection criteria for scatterplot smoothers
• Data-based interval estimation of classification error rates
• Determination of the Selection Statistics and Best Significance Level in Backward Stepwise Logistic Regression
• On the association between a random parameter and an observable
• SURE-tuned tapering estimation of large covariance matrices
• A multistage algorithm for best-subset model selection based on the Kullback-Leibler discrepancy
• Model evaluation, discrepancy function estimation, and social choice theory
• Robust estimation in regression and classification methods for large dimensional data
• A note on the generalized degrees of freedom under the \(L_{1}\) loss function
• An assumption for the development of bootstrap variants of the Akaike information criterion in mixed models
• Determination of different types of fixed effects in three-dimensional panels
• Statistical properties of convex clustering
• Density Deconvolution With Additive Measurement Errors Using Quadratic Programming

This page was built for publication: How Biased is the Apparent Error Rate of a Prediction Rule?

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3757198)