Always Valid Inference: Continuous Monitoring of A/B Tests

DOI10.1287/opre.2021.2135OpenAlexW3190343448MaRDI QIDQ5095177

Pete Koomen, David Walsh, Leonid Pekelis, Ramesh Johari

Publication date: 5 August 2022

Published in: Operations Research (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1287/opre.2021.2135

zbMATH Keywords

confidence intervalsmultiple hypothesis testing\(p\)-valuessequential hypothesis testingA/B testing

Mathematics Subject Classification ID

Statistics (62-XX)

Related Items (5)

Confidence sequences with composite likelihoods ⋮ Online multiple hypothesis testing ⋮ Game-theoretic statistics and safe anytime-valid inference ⋮ Catoni-style confidence sequences for heavy-tailed mean estimation ⋮ Generic E-Variables for Exact Sequential k-Sample Tests that allow for Optional Stopping

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Sequential analysis. Tests and confidence intervals
• The expected sample size of some tests of power one
• Approximations to the expected sample size of certain sequential tests
• Online rules for control of false discovery rate and false discovery exceedance
• α-Investing: a Procedure for Sequential Control of Expected False Discoveries
• Pure Exploration in Multi-armed Bandits Problems
• Estimation Following Sequential Tests
• Bandit Algorithms
• Semiparametric exponential families for heavy-tailed data
• Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems
• CONFIDENCE SEQUENCES FOR MEAN, VARIANCE, AND MEDIAN
• Statistical Methods Related to the Law of the Iterated Logarithm
• Sequential Tests of Statistical Hypotheses

This page was built for publication: Always Valid Inference: Continuous Monitoring of A/B Tests