Statistical Challenges in Online Controlled Experiments: A Review of A/B Testing Methodology

DOI10.1080/00031305.2023.2257237MaRDI QIDQ6955201zbMATH OpenFDO

Authors Jonathan W. Stallrich, Srijan Sengupta, Alex Deng, Ron Kohavi, Nathaniel T. Stevens

Publication date 23 July 2025

Published in The American Statistician (Search for Journal in Brave)

Copyright license Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International

Full work available at URL https://doi.org/10.1080/00031305.2023.2257237

Mathematics Subject Classification ID

Statistics (62-XX)

Cites work

• A modified sequential probability ratio test
• Always Valid Inference: Continuous Monitoring of A/B Tests
• Causal inference for statistics, social, and biomedical sciences. An introduction
• Controlling the false discovery rate via knockoffs
• D-optimal design for network A/B testing
• Discrete Sequential Boundaries for Clinical Trials
• Estimating individualized treatment rules using outcome weighted learning
• Estimating treatment effect heterogeneity in randomized program evaluation
• Estimation and Inference of Heterogeneous Treatment Effects using Random Forests
• scientific article; zbMATH DE number 2172949 (Why is no real title available?)
• scientific article; zbMATH DE number 3045589 (Why is no real title available?)
• Importance Sampling for Generalized Likelihood Ratio Procedures in Sequential Analysis
• Locally Optimal Design for A/B Tests in the Presence of Covariates and Network Dependence
• Model-assisted design of experiments in the presence of network-correlated outcomes
• Novelty and Primacy: A Long-Term Estimator for Online Experiments
• Optimal block designs for experiments on networks
• Optimal design of experiments on connected units with application to social networks
• Point estimation for adaptive trial designs. I: A methodological review
• Principal stratification in causal inference
• Recursive partitioning for heterogeneous causal effects
• Root-N-Consistent Semiparametric Regression
• Sampling-based versus design-based uncertainty in regression analysis
• Semiparametric theory and missing data.
• Sequential Tests of Statistical Hypotheses

Cited in

(5)

• Rerandomization Algorithms for Optimal Designs of Network A/B Tests
• An Economical Approach to Design Posterior Analyses
• A Review of Design of Experiments Courses Offered to Undergraduate Students at American Universities
• Genetic Algorithm-Based Bayesian Optimal Design for Network Experiments
• The Bayes e-value: testing by betting given a prior probability of the null hypothesis

This page was built for publication: Statistical Challenges in Online Controlled Experiments: A Review of A/B Testing Methodology

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6955201)