Almost the best of three worlds: risk, consistency and optional stopping for the switch criterion in nested model selection

DOI10.5705/SS.202016.0011MaRDI QIDQ4602123zbMATH OpenOpenAlexWikidataFDO

Authors Stéphanie van der Pas, Peter D. Grünwald

Publication date 26 January 2018

Published in STATISTICA SINICA (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1408.5724

zbMATH Keywords

AIC-BIC dilemmaconsistencyexponential familymodel selectionoptional stoppingpost model selection estimationswitch distributionworst-case risk

Mathematics Subject Classification ID

Bayesian problems; characterization of Bayes procedures (62C10) Statistical ranking and selection procedures (62F07)

Abstract: We study the switch distribution, introduced by Van Erven et al. (2012), applied to model selection and subsequent estimation. While switching was known to be strongly consistent, here we show that it achieves minimax optimal parametric risk rates up to a $loglogn$ factor when comparing two nested exponential families, partially confirming a conjecture by Lauritzen (2012) and Cavanaugh (2012) that switching behaves asymptotically like the Hannan-Quinn criterion. Moreover, like Bayes factor model selection but unlike standard significance testing, when one of the models represents a simple hypothesis, the switch criterion defines a robust null hypothesis test, meaning that its Type-I error probability can be bounded irrespective of the stopping rule. Hence, switching is consistent, insensitive to optional stopping and almost minimax risk optimal, showing that, Yang's (2005) impossibility result notwithstanding, it is possible to `almost' combine the strengths of AIC and Bayes factor model selection.

Recommendations

• Catching up Faster by Switching Sooner: A Predictive Approach to Adaptive Estimation with an Application to the AIC–BIC Dilemma
• An asymptotic property of model selection criteria
• Model Selection Criteria: An Investigation of Relative Accuracy, Posterior Probabilities, and Combinations of Criteria
• On the choice of a model to fit data from an exponential family
• On the convergence rate of model selection criteria

Cited in

(7)

• Safe probability
• Large-scale network motif analysis using compression
• Catching up Faster by Switching Sooner: A Predictive Approach to Adaptive Estimation with an Application to the AIC–BIC Dilemma
• Minimum description length revisited
• D-trace estimation of a precision matrix with eigenvalue control
• Optional stopping with Bayes factors: a categorization and extension of folklore results, with an application to invariant situations
• Generic E-Variables for Exact Sequential k-Sample Tests that allow for Optional Stopping

This page was built for publication: Almost the best of three worlds: risk, consistency and optional stopping for the switch criterion in nested model selection

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4602123)