On over-fitting in model selection and subsequent selection bias in performance evaluation

zbMATH Open1242.62051MaRDI QIDQ2896129FDOQ2896129

Authors: Nicola L. C. Talbot, Gavin C. Cawley 

Publication date: 13 July 2012

Published in: Journal of Machine Learning Research (JMLR) (Search for Journal in Brave)

Full work available at URL: http://www.jmlr.org/papers/v11/cawley10a.html

Recommendations

• On overfitting and post-selection uncertainty assessments
• 10.1162/153244303322753715
• On the underfitting and overfitting sets of models chosen by order selection criteria.
• The problem of over-fitting in the procedure of model selection based on structural risk minimization
• scientific article; zbMATH DE number 5957449
• On method overfitting
• Model selection: from theory to practice

zbMATH Keywords

selection biasbias-variance trade-offoverfitting

Mathematics Subject Classification ID

Classification and discrimination; cluster analysis (statistical aspects) (62H30) Learning and adaptive systems in artificial intelligence (68T05) Ridge regression; shrinkage estimators (Lasso) (62J07)

Cited In (36)

• A survey of Bayesian predictive methods for model assessment, selection and comparison
• Test Data Reuse for the Evaluation of Continuously Evolving Classification Algorithms Using the Area under the Receiver Operating Characteristic Curve
• Random projections as regularizers: learning a linear discriminant from fewer observations than dimensions
• Title not available (Why is that?)
• Projective inference in high-dimensional problems: prediction and feature selection
• Comparison of Bayesian predictive methods for model selection
• Comparing machine learning algorithms by union-free generic depth
• Smoothing regression and impact measures for accidents of traffic flows
• Efficient estimation and correction of selection-induced bias with order statistics
• Multi-population mortality forecasting using tensor decomposition
• Multi-source transfer regression via source-target pairwise segment
• Bayesian synergistic metamodeling (BSM) for physical information infused data-driven metamodeling
• Assessing Bayesian Semi‐Parametric Log‐Linear Models: An Application to Disclosure Risk Estimation
• Global resolution of the support vector machine regression parameters selection problem with LPCC
• Transfer learning based on sparse Gaussian process for regression
• Machine learning-based prediction of \(Q\)-voter model in complex networks
• Harmless overfitting: using denoising autoencoders in estimation of distribution algorithms
• 10.1162/153244303322753715
• Frequentist Model Averaging for Undirected Gaussian Graphical Models
• Hold-out strategy for selecting learning models: application to categorization subjected to presentation orders
• A comparison of collapsed Bayesian methods for probabilistic finite automata
• Kernel learning at the first level of inference
• Learning kernel logistic regression in the presence of class label noise
• Title not available (Why is that?)
• Locally linear ensemble for regression
• Learning optimized risk scores
• RNN-based deep learning for physical activity recognition using smartwatch sensors: a case study of simple and complex activity recognition
• An analysis on the relationship between uncertainty and misclassification rate of classifiers
• Two-level monotonic multistage recommender systems
• The problem of over-fitting in the procedure of model selection based on structural risk minimization
• European exchange trading funds trading with locally weighted support vector regression
• Overfitting in linear feature extraction for classification of high-dimensional image data
• Optimality of training/test size and resampling effectiveness in cross-validation
• How hard is it to pick the right model? MCS and backtest overfitting
• Feature selection and tumor classification for microarray data using relaxed Lasso and generalized multi-class support vector machine
• Feature selection based on the best-path algorithm in high dimensional graphical models

This page was built for publication: On over-fitting in model selection and subsequent selection bias in performance evaluation

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2896129)