Generic E-Variables for Exact Sequential k-Sample Tests that allow for Optional Stopping
From MaRDI portal
(Redirected from Publication:87830)
Abstract: We develop E-variables for testing whether two or more data streams come from the same source or not, and more generally, whether the difference between the sources is larger than some minimal effect size. These E-variables lead to exact, nonasymptotic tests that remain safe, i.e. keep their type-I error guarantees, under flexible sampling scenarios such as optional stopping and continuation. In special cases our E-variables also have an optimal 'growth' property under the alternative. While the construction is generic, we illustrate it through the special case of k x 2 contingency tables, where we also allow for the incorporation of different restrictions on a composite alternative. Comparison to p-value analysis in simulations and a real-world example show that E-variables, through their flexibility, often allow for early stopping of data collection, thereby retaining similar power as classical methods, while also retaining the option of extending or combining data afterwards.
Recommendations
- Exact anytime-valid confidence intervals for contingency tables and beyond
- Sequential tests controlling generalized familywise error rates
- Sequential multiple testing with generalized error control: an asymptotic optimality theory
- Multistage tests of multiple hypotheses
- Optimal stopping rules for sequential hypothesis testing
Cites work
- A Sequential Non-Parametric Multivariate Two-Sample Test
- Almost the best of three worlds: risk, consistency and optional stopping for the switch criterion in nested model selection
- Always Valid Inference: Continuous Monitoring of A/B Tests
- Bayes factors for independence in contingency tables
- CONFIDENCE SEQUENCES FOR MEAN, VARIANCE, AND MEDIAN
- E-values: calibration, combination and applications
- Efficient tracking of a growing number of experts
- Exact anytime-valid confidence intervals for contingency tables and beyond
- scientific article; zbMATH DE number 409719 (Why is no real title available?)
- scientific article; zbMATH DE number 480237 (Why is no real title available?)
- scientific article; zbMATH DE number 3062496 (Why is no real title available?)
- Likelihood, Replicability and Robbins' Confidence Sequences
- Log-optimal anytime-valid E-values
- On confidence sequences
- Present Position and Potential Developments: Some Personal Views: Statistical Theory: The Prequential Approach
- Sequential analysis. Tests and confidence intervals
- Statistical Evidence
- Statistical Methods Related to the Law of the Iterated Logarithm
- Test martingales, Bayes factors and \(p\)-values
- Time-uniform, nonparametric, nonasymptotic confidence sequences
- Valid sequential inference on probability forecast performance
Cited in
(33)- Exact anytime-valid confidence intervals for contingency tables and beyond
- E-statistics, group invariance and anytime-valid testing
- Game-theoretic statistical inference: optional sampling, universal inference, and multiple testing based on e-values. Abstracts from the workshop held May 5--10, 2024
- Rescaled Bayes factors: a class of e-variables
- Supermartingales for one-sided tests: sufficient monotone likelihood ratios are sufficient
- Authors' reply to the discussion of `safe testing'
- Zihao Wen and David L. Dowe's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Maozai Tian, Keming Yu and Jiangfeng Wang's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Andrej Srakar's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Judith ter Schure's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Christian P. Robert and Joshua Bon's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Stefano Rizzelli's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Luigi Pace and Alessandra Salvan's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Joris Mulder's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Alexander Ly's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Sander Greenland's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Neil Dey, Ryan Martin, and Jonathan P. Williams' contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Christine P. Chai's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Marco Cattaneo's contribution to the discussion of ``Safe testing by Grünwald, de Heide, and Koolen
- Joshua bon and christian P. Robert's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Vladimir Vovk's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Wenkai Xu's contribution to the discussion of `safe testing' by Grünwald, de Heide and Koolen
- Christian Hennig's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Glenn Shafer's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Thorsten Dickhaus's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Martin larsson, aaditya ramdas, and johannes Ruf's contribution to the discussion of `safe testing' by Grünwald, de heide, and koolen
- David R. Bickel's contribution to the discussion of `safe testing' by Grünwald, de Heide, and Koolen
- Seconder of the vote of thanks to Grünwald, de Heide, and Koolen and contribution to the discussion of `safe testing'
- Proposer of the vote of thanks to Grünwald, de Heide, and Koolen and contribution to the discussion of `safe testing'
- Safe testing
- A rank-based sequential test of independence
- E-values for exponential families: the general case
- safestats
This page was built for publication: Generic E-Variables for Exact Sequential k-Sample Tests that allow for Optional Stopping
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q87830)
