Rejection odds and rejection ratios: a proposal for statistical practice in testing hypotheses
From MaRDI portal
(Redirected from Publication:296932)
Abstract: Much of science is (rightly or wrongly) driven by hypothesis testing. Even in situations where the hypothesis testing paradigm is correct, the common practice of basing inferences solely on p-values has been under intense criticism for over 50 years. We propose, as an alternative, the use of the odds of a correct rejection of the null hypothesis to incorrect rejection. Both pre-experimental versions (involving the power and Type I error) and post-experimental versions (depending on the actual data) are considered. Implementations are provided that range from depending only on the p-value to consideration of full Bayesian analysis. A surprise is that all implementations -- even the full Bayesian analysis -- have complete frequentist justification. Versions of our proposal can be implemented that require only minor modifications to existing practices yet overcome some of their most severe shortcomings.
Recommendations
- Bayesian hypothesis testing: redux
- Revised standards for statistical evidence
- The use of p-values in applied research: interpretation and new trends
- Testing a Point Null Hypothesis: The Irreconcilability of P Values and Evidence
- Adaptative significance levels using optimal decision rules: balancing by weighting the error probabilities
Cites work
- scientific article; zbMATH DE number 48701 (Why is no real title available?)
- scientific article; zbMATH DE number 472940 (Why is no real title available?)
- scientific article; zbMATH DE number 3276287 (Why is no real title available?)
- A unified conditional frequentist and Bayesian test for fixed and sequential simple hypothesis testing
- Calibration of values for testing precise null hypotheses
- Could Fisher, Jeffreys and Neyman have agreed on testing? (With comments and a rejoinder).
- Default Bayes Factors for Nonnested Hypothesis Testing
- Fixed-Sample-Size Analysis of Sequential Observations
- Frequentist probability and frequentist statistics
- Revised standards for statistical evidence
- Smoothness and Convex Area Functionals—Revisited
- Statistical decision theory and Bayesian analysis. 2nd ed
- Unified Conditional Frequentist and Bayesian Testing of Composite Hypotheses
- Unified frequentist and Bayesian testing of a precise hypothesis. With comments by Dennis V. Lindley, Thomas A. Louis and David Hinkley and a rejoinder by the authors
Cited in
(53)- Inference for negativist theory using numerically computed rejection regions
- A note on type S/M errors in hypothesis testing
- Multiple Improvements of Multiple Imputation Likelihood Ratio Tests
- Replication success under questionable research practices -- a simulation study
- Bayesian Calibration of p‐Values from Fisher's Exact Test
- A classical measure of evidence for general null hypotheses
- Statistical paradises and paradoxes in big data. I: Law of large populations, big data paradox, and the 2016 US presidential election
- Benjamin, D. J., and Berger, J. O. (2019), “Three Recommendations for Improving the Use of p-Values”, The American Statistician, 73, 186–191: Comment by Foulley
- Editors' introduction to the special issue ``Bayes factors for testing hypotheses in psychological research: practical relevance and new developments
- Optional stopping with Bayes factors: a categorization and extension of folklore results, with an application to invariant situations
- How the Maximal Evidence of P-Values Against Point Null Hypotheses Depends on Sample Size
- Almost sure hypothesis testing and a resolution of the Jeffreys-Lindley paradox
- Null Hypothesis Significance Testing Interpreted and Calibrated by Estimating Probabilities of Sign Errors: A Bayes-Frequentist Continuum
- Neyman–Pearson lemma for Bayes factors
- Differentially private hypothesis testing with the subsampled and aggregated randomized response mechanism
- Time to dispense with the \(p\)-value in OR? Rationale and implications of the statement of the American Statistical Association (ASA) on \(p\)-values
- Author's reply to the discussion of `Testing by betting: a strategy for statistical and scientific communication' by Glenn Shafer
- Priyantha Wijayatunga's contribution to the discussion of `Testing by betting: a strategy for statistical and scientific communication' by Glenn Shafer
- Ruodu Wang's contribution to the discussion of `Testing by betting: a strategy for statistical and scientific communication' by Glenn Shafer
- Paul Vos's contribution to the discussion of `Testing by betting: a strategy for statistical and scientific communication' by Glenn Shafer
- Judith Ter Schure's contribution to the discussion of `Testing by betting: a strategy for statistical and scientific communication' by Glenn Shafer
- Stephen Senn's contribution to the discussion of `Testing by betting: a strategy for statistical and scientific communication' by Glenn Shafer
- Jorge Mateu's contribution to the discussion of `Testing by betting: a strategy for statistical and scientific communication' by Glenn Shafer
- Ryan Martin's contribution to the discussion of `Testing by betting: a strategy for statistical and scientific communication' by Glenn Shafer
- Nick Longford's contribution to the discussion of `Testing by betting: a strategy for statistical and scientific communication' by Glenn Shafer
- Tze Leung Lai and Anna Choi's contribution to the discussion of `Testing by betting: a strategy for statistical and scientific communication' by Glenn Shafer
- Kuldeep Kumar's contribution to the discussion of `Testing by betting: a strategy for statistical and scientific communication' by Glenn Shafer
- Chloe Krakauer and Kenneth Rice's contribution to the discussion of `Testing by betting: a strategy for statistical and scientific communication' by Glenn Shafer
- Sander Greenland's contribution to the discussion of `Testing by betting: a strategy for statistical and scientific communication' by Glenn Shafer
- Christine P. Chai's contribution to the discussion of `Testing by betting: a strategy for statistical and scientific communication' by Glenn Shafer
- A distillation of the live chat during the discussion of `Testing by betting: a strategy for statistical and scientific communication' by Glenn Shafer
- Christian Hennig's contribution to the discussion of `Testing by betting: a strategy for statistical and scientific communication' by Glenn Shafer
- Vladimir Vovk's contribution to the discussion of `Testing by betting: a strategy for statistical and scientific communication' by Glenn Shafer
- Arthur Paul Pedersen's contribution to the discussion of `Testing by betting: a strategy for statistical and scientific communication' by Glenn Shafer
- Xiao-Li Meng's contribution to the discussion of `Testing by betting: a strategy for statistical and scientific communication' by Glenn Shafer
- Peter D. Grünwald's contribution to the discussion of `Testing by betting: a strategy for statistical and scientific communication' by Glenn Shafer
- Aaditya Ramdas's contribution to the discussion of `Testing by betting: a strategy for statistical and scientific communication' by Glenn Shafer
- Barbara Osimani's contribution to the discussion of `Testing by betting: a strategy for statistical and scientific communication' by Glenn Shafer
- Harry Crane's contribution to the discussion of `Testing by betting: a strategy for statistical and scientific communication' by Glenn Shafer
- Seconder of the vote of thanks to Glenn Shafer and contribution to the discussion of `Testing by betting: a strategy for statistical and scientific communication'
- Proposer of the vote of thanks to Glenn Shafer and contribution to the discussion of `Testing by betting: a strategy for statistical and scientific communication'
- Testing by betting: a strategy for statistical and scientific communication
- A Bayes Factor for Replications of ANOVA Results
- Relative likelihood ratios for neutral comparisons of statistical tests in simulation studies
- A tutorial on bridge sampling
- scientific article; zbMATH DE number 4092545 (Why is no real title available?)
- Agnostic tests can control the type I and type II errors simultaneously
- The use of p-values in applied research: interpretation and new trends
- Putting the P-Value in its Place
- An Introduction to Second-Generation p-Values
- The impact of credit on economic growth in Vietnam: a comparison of traditional methods and the Bayes method
- Bayesian model selection in the \(\mathcal{M}\)-open setting -- approximate posterior inference and subsampling for efficient large-scale leave-one-out cross-validation via the difference estimator
- Three Recommendations for Improving the Use of p-Values
This page was built for publication: Rejection odds and rejection ratios: a proposal for statistical practice in testing hypotheses
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q296932)
