Online multiple hypothesis testing
From MaRDI portal
Abstract: Modern data analysis frequently involves large-scale hypothesis testing, which naturally gives rise to the problem of maintaining control of a suitable type I error rate, such as the false discovery rate (FDR). In many biomedical and technological applications, an additional complexity is that hypotheses are tested in an online manner, one-by-one over time. However, traditional procedures that control the FDR, such as the Benjamini-Hochberg procedure, assume that all p-values are available to be tested at a single time point. To address these challenges, a new field of methodology has developed over the past 15 years showing how to control error rates for online multiple hypothesis testing. In this framework, hypotheses arrive in a stream, and at each time point the analyst decides whether to reject the current hypothesis based both on the evidence against it, and on the previous rejection decisions. In this paper, we present a comprehensive exposition of the literature on online error rate control, with a review of key theory as well as a focus on applied examples. We also provide simulation results comparing different online testing algorithms and an up-to-date overview of the many methodological extensions that have been proposed.
Cites work
- α-Investing: a Procedure for Sequential Control of Expected False Discoveries
- A Direct Approach to False Discovery Rates
- Always Valid Inference: Continuous Monitoring of A/B Tests
- FDR control with pseudo-gatekeeping based on a possibly data driven order of the hypotheses
- Generalized α-investing: Definitions, Optimality Results and Application to Public Databases
- k-FWER control without \(p \)-value adjustment, with application to detection of genetic determinants of multiple sclerosis in Italian twins
- Multiple testing when many \(p\)-values are uniformly conservative, with application to testing qualitative interaction in educational interventions
- On the power of some sequential multiple testing procedures
- Online rules for control of false discovery rate and false discovery exceedance
- Rectangular Confidence Regions for the Means of Multivariate Normal Distributions
- Simultaneous high-probability bounds on the false discovery proportion in structured, regression and online settings
- Simultaneous Testing of Grouped Hypotheses: Finding Needles in Multiple Haystacks
- Structure–Adaptive Sequential Testing for Online False Discovery Rate Control
- Time-uniform, nonparametric, nonasymptotic confidence sequences
Cited in
(5)- The online closure principle
- An exhaustive ADDIS principle for online FWER control
- ADDIS-graphs for online error control with application to platform trials
- Discussion of “Poisson-FOCuS: An Efficient Online Method for Detecting Count Bursts with Application to Gamma Ray Burst Detection”
- Bayesian nonparametric sensitivity analysis of multiple test procedures under dependence
This page was built for publication: Online multiple hypothesis testing
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6145148)
