How Biased is the Apparent Error Rate of a Prediction Rule?

DOI10.2307/2289236MaRDI QIDQ3757198zbMATH OpenOpenAlexFDO

Authors Bradley Efron

Publication date 1986

Full work available at URL https://doi.org/10.2307/2289236

zbMATH Keywords

AICAkaike criteriumbootstrapcross validationdownward biaserror rate of prediction ruleexamplegeneral exponential family linear modelsgeneralized linear modellogistic modelMallow's Cpmeasures of prediction errorsNumerical resultsoptimismprediction errorsprediction modelunderestimation

Mathematics Subject Classification ID

Point estimation (62F10) Parametric inference (62F99) Linear regression; mixed models (62J05) Linear inference, regression (62J99)

Recommendations

• Estimating the Error Rate of a Prediction Rule: Improvement on Cross-Validation
• On the estimation of prediction errors in logistic regression models
• Overestimation of the receiver operating characteristic curve for logistic regression
• On the biases of error estimators in prediction problems
• Assessing the performance of an allocation rule

Cited in

(only showing first 100 items - show all)

• Bootstrap variants of the Akaike information criterion for mixed model selection
• Estimation of the conditional risk in classification: the swapping method
• Asymptotic bootstrap corrections of AIC for linear regression models
• Nearly unbiased variable selection under minimax concave penalty
• On model selection via stochastic complexity in robust linear regression
• Appropriate penalties in the final prediction error criterion: A decision theoretic approach
• Using specially designed exponential families for density estimation
• Reluctant generalized additive modeling
• Is \(C_{p}\) an empirical Bayes method for smoothing parameter choice?
• Flexible and Interpretable Models for Survival Data
• P-splines with an \(\ell_1\) penalty for repeated measures
• Tuning parameter selection in sparse regression modeling
• Model selection criteria based on cross-validatory concordance statistics
• Extending AIC to best subset regression
• Smoothing spline ANOVA models for large data sets with Bernoulli observations and the randomized GACV.
• Selection criteria for scatterplot smoothers
• Least angle regression. (With discussion)
• Evaluating the impact of exploratory procedures in regression prediction: A pseudosample approach
• On the biases of error estimators in prediction problems
• Model selection by resampling penalization
• Asymptotic optimality of full cross-validation for selecting linear regression models
• A large-sample model selection criterion based on Kullback's symmetric divergence
• Automated data-driven selection of the hyperparameters for total-variation-based texture segmentation
• Degrees of freedom for off-the-grid sparse estimation
• Degrees of freedom and model selection for \(k\)-means clustering
• A significance test for the lasso
• Discussion: ``A significance test for the lasso
• Rejoinder: ``A significance test for the lasso
• Local behavior of sparse analysis regularization: applications to risk estimation
• Distance-based linear discriminant analysis for interval-valued data
• Additive models with trend filtering
• Assessing prediction error at interpolation and extrapolation points
• Compressed covariance estimation with automated dimension learning
• Optimality of training/test size and resampling effectiveness in cross-validation
• On the predictive risk in misspecified quantile regression
• Bayesian comparison of latent variable models: conditional versus marginal likelihoods
• Comparing and selecting spatial predictors using local criteria
• SURE-tuned tapering estimation of large covariance matrices
• Asymptotic properties of a double penalized maximum likelihood estimator in logistic regres\-sion
• The degrees of freedom of partly smooth regularizers
• Efficient regularized isotonic regression with application to gene-gene interaction search
• Measuring the prediction error. A comparison of cross-validation, bootstrap and covariance penalty methods
• Modeling strategies in longitudinal data analysis: covariate, variance function and correlation structure selection
• Adapting to unknown sparsity by controlling the false discovery rate
• Bayesian nonparametric model selection and model testing
• A model search procedure for hierarchical models
• Sparse estimation via nonconcave penalized likelihood in factor analysis model
• Modelling of insurers' rating determinants. An application of machine learning techniques and statistical models
• Subspace information criterion for model selection
• Low complexity regularization of linear inverse problems
• Variable selection in canonical discriminant analysis for family studies
• Model selection for factorial Gaussian graphical models with an application to dynamic regulatory networks
• Prediction-based structured variable selection through the receiver operating characteristic curves
• Multiple group linear discriminant analysis: robustness and error rate
• Evaluation of generalized degrees of freedom for sparse estimation by replica method
• From Fixed-X to Random-X Regression: Bias-Variance Decompositions, Covariance Penalties, and Prediction Error Estimation
• Discussion of “From Fixed-X to Random-X Regression: Bias-Variance Decompositions, Covariance Penalties, and Prediction Error Estimation”
• Compressed and Penalized Linear Regression
• Combining neural networks for function approximation under conditions of sparse data: the biased regression approach
• Quantifying the Predictive Performance of Prognostic Models for Censored Survival Data with Time-Dependent Covariates
• Maximizing proportions of correct classifications in binary logistic regression
• Efficient Computation and Model Selection for the Support Vector Regression
• New aspects of Bregman divergence in regression and classification with parametric and nonparametric estimation
• Inference after variable selection using restricted permutation methods
• A lasso for hierarchical interactions
• Un critère de choix de variables en analyse en composantes principales fondé sur des modèles graphiques gaussiens particuliers
• Determination of the best significance level in forward stepwise logistic regression
• Degrees of freedom in lasso problems
• Data-based interval estimation of classification error rates
• Assessing the performance of data assimilation algorithms which employ linear error feedback
• Prediction Using Partly Conditional Time‐Varying Coefficients Regression Models
• On the estimation of prediction errors in logistic regression models
• Are ordinal models useful for classification? a revised analysis
• A study on tuning parameter selection for the high-dimensional lasso
• Regular, median and Huber cross‐validation: A computational comparison
• A non-convex regularization approach for stable estimation of loss development factors
• A Pliable Lasso
• Density Deconvolution With Additive Measurement Errors Using Quadratic Programming
• Bootstrap estimation and model selection for multivariate normal mixtures using parallel computing with graphics processing units
• Randomization as regularization: a degrees of freedom explanation for random forest success
• Estimating the accuracy of (local) cross-validation via randomised GCV choices in kernel or smoothing spline regression
• Excess optimism: how biased is the apparent error of an estimator tuned by SURE?
• On the optimism correction of the area under the receiver operating characteristic curve in logistic prediction models
• Degrees of freedom in low rank matrix estimation
• Prediction Error Estimation Under Bregman Divergence for Non‐Parametric Regression and Classification
• Determination of the Selection Statistics and Best Significance Level in Backward Stepwise Logistic Regression
• Ideal point discriminant analysis
• Determination of different types of fixed effects in three-dimensional panels
• Reconceptualizing the p -value from a likelihood ratio test: a probabilistic pairwise comparison of models based on Kullback-Leibler discrepancy measures
• High-Dimensional Spatial Quantile Function-on-Scalar Regression
• Cross-Validation for Correlated Data
• The asymptotic distribution of the proportion of correct classifications for a holdout sample in logistic regression
• A regression model selection criterion based on bootstrap bumping for use with resistant fitting.
• Statistical significance of the Netflix challenge
• Recent developments in bootstrap methodology
• Discussion: ``A significance test for the lasso
• Discussion: ``A significance test for the lasso
• Discussion: ``A significance test for the lasso
• Discussion: ``A significance test for the lasso
• Discussion: ``A significance test for the lasso

This page was built for publication: How Biased is the Apparent Error Rate of a Prediction Rule?

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3757198)