Rejection odds and rejection ratios: a proposal for statistical practice in testing hypotheses
From MaRDI portal
(Redirected from Publication:296932)
Abstract: Much of science is (rightly or wrongly) driven by hypothesis testing. Even in situations where the hypothesis testing paradigm is correct, the common practice of basing inferences solely on p-values has been under intense criticism for over 50 years. We propose, as an alternative, the use of the odds of a correct rejection of the null hypothesis to incorrect rejection. Both pre-experimental versions (involving the power and Type I error) and post-experimental versions (depending on the actual data) are considered. Implementations are provided that range from depending only on the p-value to consideration of full Bayesian analysis. A surprise is that all implementations -- even the full Bayesian analysis -- have complete frequentist justification. Versions of our proposal can be implemented that require only minor modifications to existing practices yet overcome some of their most severe shortcomings.
Recommendations
- Bayesian hypothesis testing: redux
- Revised standards for statistical evidence
- The use of \(p\)-values in applied research: interpretation and new trends
- Testing a Point Null Hypothesis: The Irreconcilability of P Values and Evidence
- Adaptative significance levels using optimal decision rules: balancing by weighting the error probabilities
Cites work
- scientific article; zbMATH DE number 48701 (Why is no real title available?)
- scientific article; zbMATH DE number 472940 (Why is no real title available?)
- scientific article; zbMATH DE number 3276287 (Why is no real title available?)
- A unified conditional frequentist and Bayesian test for fixed and sequential simple hypothesis testing
- Calibration of \(\rho \) values for testing precise null hypotheses
- Could Fisher, Jeffreys and Neyman have agreed on testing? (With comments and a rejoinder).
- Default Bayes Factors for Nonnested Hypothesis Testing
- Fixed-Sample-Size Analysis of Sequential Observations
- Frequentist probability and frequentist statistics
- Revised standards for statistical evidence
- Smoothness and Convex Area Functionals—Revisited
- Statistical decision theory and Bayesian analysis. 2nd ed
- Unified Conditional Frequentist and Bayesian Testing of Composite Hypotheses
- Unified frequentist and Bayesian testing of a precise hypothesis. With comments by Dennis V. Lindley, Thomas A. Louis and David Hinkley and a rejoinder by the authors
Cited in
(26)- Bayesian Calibration of p‐Values from Fisher's Exact Test
- Relative likelihood ratios for neutral comparisons of statistical tests in simulation studies
- Inference for negativist theory using numerically computed rejection regions
- A note on type S/M errors in hypothesis testing
- Optional stopping with Bayes factors: a categorization and extension of folklore results, with an application to invariant situations
- The impact of credit on economic growth in Vietnam: a comparison of traditional methods and the Bayes method
- Agnostic tests can control the type I and type II errors simultaneously
- Multiple Improvements of Multiple Imputation Likelihood Ratio Tests
- Replication success under questionable research practices -- a simulation study
- How the Maximal Evidence of P-Values Against Point Null Hypotheses Depends on Sample Size
- Benjamin, D. J., and Berger, J. O. (2019), “Three Recommendations for Improving the Use of p-Values”, The American Statistician, 73, 186–191: Comment by Foulley
- Bayesian model selection in the \(\mathcal{M}\)-open setting -- approximate posterior inference and subsampling for efficient large-scale leave-one-out cross-validation via the difference estimator
- Statistical paradises and paradoxes in big data. I: Law of large populations, big data paradox, and the 2016 US presidential election
- An Introduction to Second-Generation p-Values
- Almost sure hypothesis testing and a resolution of the Jeffreys-Lindley paradox
- A tutorial on bridge sampling
- scientific article; zbMATH DE number 4092545 (Why is no real title available?)
- Three Recommendations for Improving the Use of p-Values
- A classical measure of evidence for general null hypotheses
- Neyman–Pearson lemma for Bayes factors
- Null Hypothesis Significance Testing Interpreted and Calibrated by Estimating Probabilities of Sign Errors: A Bayes-Frequentist Continuum
- Time to dispense with the \(p\)-value in OR? Rationale and implications of the statement of the American Statistical Association (ASA) on \(p\)-values
- Editors' introduction to the special issue ``Bayes factors for testing hypotheses in psychological research: practical relevance and new developments
- A Bayes Factor for Replications of ANOVA Results
- The use of \(p\)-values in applied research: interpretation and new trends
- Putting the P-Value in its Place
This page was built for publication: Rejection odds and rejection ratios: a proposal for statistical practice in testing hypotheses
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q296932)
