False confidence, non-additive beliefs, and valid statistical inference
From MaRDI portal
Publication:2302767
Abstract: Statistics has made tremendous advances since the times of Fisher, Neyman, Jeffreys, and others, but the fundamental and practically relevant questions about probability and inference that puzzled our founding fathers remain unanswered. To bridge this gap, I propose to look beyond the two dominating schools of thought and ask the following three questions: what do scientists need out of statistics, do the existing frameworks meet these needs, and, if not, how to fill the void? To the first question, I contend that scientists seek to convert their data, posited statistical model, etc., into calibrated degrees of belief about quantities of interest. To the second question, I argue that any framework that returns additive beliefs, i.e., probabilities, necessarily suffers from {em false confidence}---certain false hypotheses tend to be assigned high probability---and, therefore, risks systematic bias. This reveals the fundamental importance of {em non-additive beliefs} in the context of statistical inference. But non-additivity alone is not enough so, to the third question, I offer a sufficient condition, called {em validity}, for avoiding false confidence, and present a framework, based on random sets and belief functions, that provably meets this condition. Finally, I discuss characterizations of p-values and confidence intervals in terms of valid non-additive beliefs, which imply that users of these classical procedures are already following the proposed framework without knowing it.
Recommendations
Cites work
- scientific article; zbMATH DE number 3136643 (Why is no real title available?)
- scientific article; zbMATH DE number 3945130 (Why is no real title available?)
- scientific article; zbMATH DE number 46364 (Why is no real title available?)
- scientific article; zbMATH DE number 48344 (Why is no real title available?)
- scientific article; zbMATH DE number 3522963 (Why is no real title available?)
- scientific article; zbMATH DE number 3551712 (Why is no real title available?)
- scientific article; zbMATH DE number 3560492 (Why is no real title available?)
- scientific article; zbMATH DE number 1220667 (Why is no real title available?)
- scientific article; zbMATH DE number 627769 (Why is no real title available?)
- scientific article; zbMATH DE number 3434895 (Why is no real title available?)
- scientific article; zbMATH DE number 2166302 (Why is no real title available?)
- scientific article; zbMATH DE number 947414 (Why is no real title available?)
- scientific article; zbMATH DE number 3441432 (Why is no real title available?)
- scientific article; zbMATH DE number 1825523 (Why is no real title available?)
- scientific article; zbMATH DE number 1911984 (Why is no real title available?)
- scientific article; zbMATH DE number 3433221 (Why is no real title available?)
- scientific article; zbMATH DE number 5586074 (Why is no real title available?)
- scientific article; zbMATH DE number 3230247 (Why is no real title available?)
- scientific article; zbMATH DE number 3263743 (Why is no real title available?)
- scientific article; zbMATH DE number 3271180 (Why is no real title available?)
- scientific article; zbMATH DE number 3189754 (Why is no real title available?)
- scientific article; zbMATH DE number 3195782 (Why is no real title available?)
- scientific article; zbMATH DE number 3046453 (Why is no real title available?)
- A Statistical Inference Course Based on p-Values
- A mathematical theory of evidence turns 40
- A mathematical theory of hints. An approach to the Dempster-Shafer theory of evidence
- A more powerful test for comparing two Poisson means
- A note on p-values interpreted as plausibilities
- Allocations of probability
- An Example of Wide Discrepancy Between Fiducial and Confidence Intervals
- An Introduction to Random Sets
- Assessing the Statistical Analyses Used in Basic and Applied Social Psychology After Their p-Value Ban
- Asymptotically minimax empirical Bayes estimation of a sparse normal mean vector
- Bayes theorem in the 21st century
- Bayes' theorem for Choquet capacities
- Bayes, reproducibility and the quest for truth
- Bayesianly justifiable and relevant frequency calculations for the applied statistician
- Calibrating general posterior credible regions
- Classic works on the Dempster-Shafer theory of belief functions
- Conditional inferential models: combining information for prior-free probabilistic inference
- Confidence and Likelihood*
- Confidence distribution, the frequentist distribution estimator of a parameter: a review
- Confidence, Likelihood, Probability
- Decision-making with belief functions: a review
- Default Priors for Bayesian and Frequentist Inference
- Default reasoning and possibility theory
- Discussion
- Exact and efficient inference for partial Bayes problems
- Exact prior-free probabilistic inference on the heritability coefficient in a linear mixed model
- Extreme value theory. An introduction.
- Fiducial Generalized Confidence Intervals
- Fisher's “Inverse Probability” of 1930
- Frequency-calibrated belief functions: review and new insights
- Frequentist statistics as a theory of inductive inference
- Further Examples of Inconsistencies in the Fiducial Argument
- Generalized Confidence Intervals
- Generalized fiducial inference for ultrahigh-dimensional regression
- Generalized fiducial inference for wavelet regression
- Generalized fiducial inference via discretization
- Gibbs posterior inference on the minimum clinically important difference
- Inference about constrained parameters using the elastic belief method
- Inferential Models: A Framework for Prior-Free Posterior Probabilistic Inference
- Inferential models. Reasoning with uncertainty
- Integrated likelihood methods for eliminating nuisance parameters. (With comments and a rejoinder).
- Interval estimation for a binomial proportion. (With comments and a rejoinder).
- Is Bayes posterior just quick and dirty confidence?
- Marginal Inferential Models: Prior-Free Probabilistic Inference on Interest Parameters
- Mathematical foundations for a theory of confidence structures
- Maximum Likelihood: An Introduction
- Measuring statistical evidence using relative belief
- Minimax tests and the Neyman-Pearson lemma for capacities
- New Methods for Reasoning Towards Posterior Distributions Based on Sample Data
- On Some Principles of Statistical Inference
- On an inferential model construction using generalized associations
- On random sets and belief functions
- On rereading R. A. Fisher
- On the Birnbaum argument for the strong likelihood principle
- On the Difficulties Inherent in Fisher's Fiducial Argument
- On the Foundations of Statistical Inference
- Pivotal Models and the Fiducial Argument
- Practical Solutions of the Behrens-Fisher Problem
- Principles of Statistical Inference
- Principles of uncertainty
- Prior envelopes based on belief functions
- Probabilistic foundations of statistical network analysis
- R. A. Fisher and the fiducial argument
- R. A. Fisher in the 21st century. Invited paper presented at the 1996 R. A. Fisher lecture. (With comments).
- R. A. Fisher on Bayes and Bayes' theorem
- R. A. Fisher on the history of inverse probability. With comments by Robin L. Plackett and G. A. Barnard and a rejoinder by the author
- R. A. Fisher's fiducial argument and Bayes' theorem
- Response to the ASA’s Statement on p-Values: Context, Process, and Purpose
- Robust Statistics
- Safe probability
- Sampling and Bayes' Inference in Scientific Modelling and Robustness
- Satellite conjunction analysis and the false confidence theorem
- Some Impossibility Theorems in Econometrics With Applications to Structural and Dynamic Models
- Statistical Inference as Severe Testing
- Symmetric Measures on Cartesian Products
- THE SIGNIFICANCE OF THE DIFFERENCE BETWEEN TWO MEANS WHEN THE POPULATION VARIANCES ARE UNEQUAL
- Testing a Point Null Hypothesis: The Irreconcilability of P Values and Evidence
- The ASA Statement on p-Values: Context, Process, and Purpose
- The Behrens-Fisher problem revisited: a Bayes-frequentist synthesis
- The Dempster-Shafer calculus for statisticians
- The IM-based method for testing the non-inferiority of odds ratio in matched-pairs design
- The Methodology of Scientific Research Programmes
- The nonexistence of \(100(1-\alpha)\%\) confidence sets of finite expected diameter in errors-in-variables and related models
- The sources of Kolmogorov's \textit{Grundbegriffe}
- Theory of Random Sets
- Theory of capacities
- Three enigmatic examples and inference from likelihood
- Towards a unified theory of imprecise probability
- Two classes of group divisible partial diallel crosses
- Understanding uncertainty
- Upper and Lower Probabilities Generated by a Random Closed Interval
- Upper and Lower Probabilities Induced by a Multivalued Mapping
- Upper and Lower Probability Inferences for Families of Hypotheses with Monotone Density Ratios
- What did Fisher mean by inverse probability in 1912--1922?
- What does the proof of Birnbaum's theorem prove?
- What is a statistical model? (With comments and rejoinder).
Cited in
(23)- Valid inferential models for prediction in supervised learning problems
- Game-theoretic statistical inference: optional sampling, universal inference, and multiple testing based on e-values. Abstracts from the workshop held May 5--10, 2024
- Special issue from the 5th international conference on belief functions (BELIEF 2018)
- Gibbs posterior inference on multivariate quantiles
- Belief functions induced by random fuzzy sets: a general framework for representing uncertain and fuzzy evidence
- Model-free posterior inference on the area under the receiver operating characteristic curve
- Valid model-free prediction of future insurance claims
- Towards a Theory of Valid Inferential Models with Partial Prior Information
- Generalized inferential models for censored data
- Confidence in confidence distributions!
- A possibility-theoretic solution to Basu's Bayesian-frequentist via media
- Verified propagation of imprecise probabilities in non-linear ODEs
- Distribution-free inferential models: achieving finite-sample valid probabilistic inference, with emphasis on quantile regression
- Model misspecification, Bayesian versus credibility estimation, and Gibbs posteriors
- A comparison of learning rate selection methods in generalized Bayesian inference
- Direct and approximately valid probabilistic inference on a class of statistical functionals
- An inferential model-based method for testing homogeneity of several variances against tree-ordered alternatives
- Satellite conjunction analysis and the false confidence theorem
- Validity, consonant plausibility measures, and Conformal prediction
- New two-sided confidence intervals for binomial inference derived using Walley's imprecise posterior likelihood as a test statistic
- A practical strategy for valid partial prior-dependent possibilistic inference
- Games of incomplete information: a framework based on belief functions
- Nonadditive probabilities in statistics
This page was built for publication: False confidence, non-additive beliefs, and valid statistical inference
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2302767)
