Combining p-values via averaging

DOI10.1093/BIOMET/ASAA027zbMATH Open1457.62078arXiv1212.4966OpenAlexW3035249419MaRDI QIDQ5145696FDOQ5145696

Ruodu Wang, Vladimir Vovk

Publication date: 21 January 2021

Published in: Biometrika (Search for Journal in Brave)

Abstract: This paper proposes general methods for the problem of multiple testing of a single hypothesis, with a standard goal of combining a number of p-values without making any assumptions about their dependence structure. An old result by R"uschendorf and, independently, Meng implies that the p-values can be combined by scaling up their arithmetic mean by a factor of 2 (and no smaller factor is sufficient in general). A similar result about the geometric mean (Mattner) replaces 2 by $e$. Based on more recent developments in mathematical finance, specifically, robust risk aggregation techniques, we extend these results to generalized means; in particular, we show that $K$ p-values can be combined by scaling up their harmonic mean by a factor of $lnK$ (asymptotically as $Koinfty$). This leads to a generalized version of the Bonferroni-Holm procedure. We also explore methods using weighted averages of p-values. Finally, we discuss the efficiency of various methods of combining p-values and how to choose a suitable method in light of data and prior information.

Full work available at URL: https://arxiv.org/abs/1212.4966

Recommendations

• A systematic comparison of methods for combining \(p\)-values from independent tests
• Combining p-values using order-based methods
• Admissible ways of merging \(p\)-values under arbitrary dependence
• On the optimally weighted \(z\)-test for combining probabilities from independent studies
• Averaging \(p\)-values under exchangeability

zbMATH Keywords

hypothesis testingmultiple hypothesis testingmultiple testing of a single hypothesisrobust risk aggregation

Mathematics Subject Classification ID

Parametric hypothesis testing (62F03) Multivariate distribution of statistics (62H10) Inequalities; stochastic orderings (60E15) Robustness and adaptive procedures (parametric inference) (62F35) Paired and multiple comparisons; multiple testing (62J15)

Cited In (29)

• Combining independent p-values in replicability analysis: a comparative study
• Post-clustering difference testing: valid inference and practical considerations with applications to ecological and biological data
• Heavy-Tailed Distribution for Combining Dependent P-Values With Asymptotic Robustness
• A STATISTICAL METHODOLOGY FOR ASSESSING THE MAXIMAL STRENGTH OF TAIL DEPENDENCE
• Distribution-Free Prediction Sets for Two-Layer Hierarchical Models
• Averaging \(p\)-values under exchangeability
• Admissible ways of merging \(p\)-values under arbitrary dependence
• Title not available (Why is that?)
• Large‐scale simultaneous inference under dependence
• A unified treatment of multiple testing with prior knowledge using the p-filter
• Sums of standard uniform random variables
• Sequentially valid tests for forecast calibration
• What is a Randomization Test?
• Combining p-values for Multivariate Predictive Ability Testing
• Order selection with confidence for finite mixture models
• Statistical inference with large-scale trait imputation
• Asymptotic Distribution-Free Independence Test for High-Dimension Data
• Testing with \(\mathrm{p}^*\)-values: between p-values, mid p-values, and e-values
• Confidence and discoveries with \(e\)-values
• Trade-off Between Validity and Efficiency of Merging P-Values Under Arbitrary Dependence
• Permutation tests using arbitrary permutation distributions
• Training-conditional coverage for distribution-free predictive inference
• Testing partial conjunction hypotheses under dependency, with applications to meta-analysis
• Testing randomness online
• Multi Split Conformal Prediction
• E-values: calibration, combination and applications
• Ordering and inequalities for mixtures on risk aggregation
• On Samuel's \(p\)-value model and the Simes test under dependence
• Multi-split conformal prediction via Cauchy aggregation

This page was built for publication: Combining \(p\)-values via averaging

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5145696)