Practical valid inferences for the two-sample binomial problem

DOI10.1214/21-SS131zbMATH Open1490.62064arXiv1904.05416OpenAlexW2939971573MaRDI QIDQ76638FDOQ76638

Authors: Sally A. Hunsberger, Sally A. Hunsberger, Michael P. Fay 

Publication date: 1 January 2021

Published in: Statistics Surveys (Search for Journal in Brave)

Abstract: Our interest is whether two binomial parameters differ, which parameter is larger, and by how much. This apparently simple problem was addressed by Fisher in the 1930's, and has been the subject of many review papers since then. Yet there continues to be new work on this issue and no consensus solution. Previous reviews have focused primarily on testing and the properties of validity and power, or primarily on confidence intervals, their coverage, and expected length. Here we evaluate both. For example, we consider whether a p-value and its matching confidence interval are compatible, meaning that the p-value rejects at level $alpha$ if and only if the $1-alpha$ confidence interval excludes all null parameter values. For focus, we only examine non-asymptotic inferences, so that most of the p-values and confidence intervals are valid (i.e., exact) by construction. Within this focus, we review different methods emphasizing many of the properties and interpretational aspects we desire from applied frequentist inference: validity, accuracy, good power, equivariance, compatibility, coherence, and parameterization and direction of effect. We show that no one method can meet all the desirable properties and give recommendations based on which properties are given more importance.

Full work available at URL: https://arxiv.org/abs/1904.05416

Recommendations

• A cautionary note on exact unconditional inference for a difference between two independent binomial proportions
• Resolving the conflict over fisher's exact test
• Contemporary frequentist views of the \(2\times 2\) binomial trial
• The exact unconditional \(z\)-pooled test for equality of two binomial probabilities: optimal choice of the Berger and Boos confidence coefficient
• Exact tests for equality of two proportions: fisher v. bayes

zbMATH Keywords

\(2\times 2\) tableBarnard's testFisher's exact testunconditional exact test

Mathematics Subject Classification ID

Parametric hypothesis testing (62F03) Parametric tolerance and confidence regions (62F25)

Cites Work

• Small-sample comparisons of confidence intervals for the difference of two independent binomial proportions
• Choosing the optimal unconditioned test for comparing two independent proportions
• The efficiency of Buehler confidence limits.
• Information, for testing the equality of two probabilities, from the margins of the \(2 \times 2\) table.
• Asymptotical tests in \(2\times 2\) comparative trials: (unconditional approach).
• Contemporary frequentist views of the \(2\times 2\) binomial trial
• The equivalence of the mid \(p\)-value and the expected \(p\)-value for testing equality of two balanced binomial proportions
• On construction of the smallest one-sided confidence interval for the difference of two proportions
• Computation of exact confidence limits from discrete data
• Improved Buehler limits based on refined designated statistics
• Confidence curves and improved exact confidence intervals for discrete distributions
• An optimality theory for mid \(p\)-values in \(2\times 2\) contingency tables
• \(UMP(U)\)-tests for a binomial parameter: A paradox
• Exact confidence intervals for the relative risk and the odds ratio
• Exact \(P\)-values for discrete models obtained by estimation and maximization
• Confidence Intervals for Proportions and Related Measures of Effect Size
• On small-sample confidence intervals for parameters in discrete distributions
• A cautionary note on exact unconditional inference for a difference between two independent binomial proportions
• The minimaxity of the mid \(P\)-value under linear and squared loss functions
• Statistics in Epidemiology: The Case-Control Study
• Confidence intervals that match Fisher's exact or Blaker's exact tests
• Combining one-sample confidence procedures for inference in the two-sample case
• Test of Significance for 2 × 2 Contingency Tables
• Computing an Exact Confidence Interval for the Common Odds Ratio in Several 2 × 2 Contingency Tables
• Small-Sample Confidence Intervals for p 1 - p 2 and p 1 /p 2 in 2 × 2 Contingency Tables
• Unconditional Non-Asymptotic One-Sided Tests for Independent Binomial Proportions When the Interest Lies in Showing Non-Inferiority and/or Superiority
• Small sample confidence intervals for the difference,ratio and odds ratio of two success probabilities
• P Values Maximized Over a Confidence Set for the Nuisance Parameter
• Table of percentage points of the behrens-fisher distribution
• On the Optimality and Limitations of Buehler Bounds
• Test‐Based Exact Confidence Intervals for the Difference of Two Binomial Proportions
• Confidence distribution, the frequentist distribution estimator of a parameter: a review
• Problems with existing procedures to calculate exact unconditional P-values for non-inferiority/superiority and confidence intervals for two binomials and how to resolve them
• Exact Analysis of Discrete Data
• Proving non-inferiority or equivalence of two treatments with dichotomous endpoints using exact methods
• Sampling Methods for Wallenius' and Fisher's Noncentral Hypergeometric Distributions
• Testing Statistical Hypotheses
• Raised conditional level of significance for the 2 × 2‐table when testing the equality of two probabilities
• Unconditional small-sample confidence intervals for the odds ratio
• Univariate Discrete Distributions
• THE MEANING OF A SIGNIFICANCE LEVEL
• A New Test for 2 × 2 Tables
• Exact solutions to the Behrens - Fisher problem: asymptotically optimal and finite sample efficient choice among

Cited In (7)

• The two-sample problem via relative belief ratio
• Optimal confidence interval for the difference between proportions
• Design of randomized clinical trials with a binary endpoint: conditional versus unconditional analyses of a two-by-two table
• Are the tests overpowered or underpowered? A unified solution to correctly specify type I errors in design of clinical trials for two sample proportions
• Plausibility functions and exact frequentist inference
• exact2x2
• Interpreting \(p\)-values and confidence intervals using well-calibrated null preference priors

Uses Software

• StatXact
• SAS
• exactci
• SAS/STAT
• PROC FREQ
• exact2x2

This page was built for publication: Practical valid inferences for the two-sample binomial problem

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q76638)