Too good to be true: when overwhelming evidence fails to convince

DOI10.1098/RSPA.2015.0748zbMATH Open1373.62026arXiv1601.00900OpenAlexW2232134439WikidataQ51355243 ScholiaQ51355243MaRDI QIDQ5363533FDOQ5363533

Authors: Lachlan J. Gunn, François Chapeau-Blondeau, Mark D. McDonnell, B. R. Davis, Andrew Allison, Derek Abbott 

Publication date: 29 September 2017

Published in: Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences (Search for Journal in Brave)

Abstract: Is it possible for a large sequence of measurements or observations, which support a hypothesis, to counterintuitively decrease our confidence? Can unanimous support be too good to be true? The assumption of independence is often made in good faith, however rarely is consideration given to whether a systemic failure has occurred. Taking this into account can cause certainty in a hypothesis to decrease as the evidence for it becomes apparently stronger. We perform a probabilistic Bayesian analysis of this effect with examples based on (i) archaeological evidence, (ii) weighing of legal evidence, and (iii) cryptographic primality testing. We find that even with surprisingly low systemic failure rates high confidence is very difficult to achieve and in particular we find that certain analyses of cryptographically-important numerical tests are highly optimistic, underestimating their false-negative rate by as much as a factor of $2^{80}$.

Full work available at URL: https://arxiv.org/abs/1601.00900

Recommendations

• Too Odd (Not) to Be True? A Reply to Olsson
• The corroboration paradox
• Reconciling probability theory and coherentism
• The variety-of-evidence thesis: a Bayesian exploration of its surprising failures
• On the Probability of Observing Misleading Statistical Evidence

Mathematics Subject Classification ID

Foundations and philosophical topics in statistics (62A01) Optimal statistical designs (62K05) Bayesian problems; characterization of Bayes procedures (62C10)

Cited In (1)

• The Market for Quacks

This page was built for publication: Too good to be true: when overwhelming evidence fails to convince

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5363533)