Comparative Probability Metrics: Using Posterior Probabilities to Account for Practical Equivalence in A/B tests

DOI10.1080/00031305.2021.2000495MaRDI QIDQ5050831zbMATH OpenOpenAlexFDO

Authors Nathaniel T. Stevens, Luke Hagar

Publication date 18 November 2022

Published in The American Statistician (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1080/00031305.2021.2000495

zbMATH Keywords

A/B testingBayesian estimationdesign and analysis of experimentsMarkov chain Monte Carlopractical significancetwo-group comparisons

Mathematics Subject Classification ID

Statistics (62-XX)

Cites work

• p-Values, Bayes Factors, and Sufficiency
• A Proposed Hybrid Effect Size Plus p-Value Criterion: Empirical Evidence Supporting its Use
• Abandon Statistical Significance
• An Introduction to Second-Generation p-Values
• Bayes Factors
• Blending Bayesian and Classical Tools to Define Optimal Sample-Size-Dependent Significance Levels
• Coup de Grâce for a Tough Old Bull: “Statistically Significant” Expires
• How Large Are Your G-Values? Try Gosset’s Guinnessometrics When a Little “p” Is Not Enough
• Moving to a World Beyond “p < 0.05”
• Moving Towards the Post p < 0.05 Era via the Analysis of Credibility
• Testing Statistical Hypotheses of Equivalence and Noninferiority
• The p-Value Requires Context, Not a Threshold
• The ASA Statement on p-Values: Context, Process, and Purpose
• The False Positive Risk: A Proposal Concerning What to Do About p-Values
• Three Recommendations for Improving the Use of p-Values
• ValidP-Values Behave Exactly as They Should: Some Misleading Criticisms ofP-Values and Their Resolution WithS-Values

Cited in

(4)

• Assessing Measurement System Agreement in the Presence of Reproducibility and Repeatability
• An Economical Approach to Design Posterior Analyses
• A Review of Design of Experiments Courses Offered to Undergraduate Students at American Universities
• Fast power curve approximation for posterior analyses

Describes a project that uses

Uses Software

• BayesDA
• rjags
• JAGS

This page was built for publication: Comparative Probability Metrics: Using Posterior Probabilities to Account for Practical Equivalence in A/B tests

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5050831)