On Suspicious Coincidences and Pointwise Mutual Information

DOI10.1162/NECO_A_01533MaRDI QIDQ5044169zbMATH OpenOpenAlexFDO

Authors Christopher K. I. Williams

Publication date 24 October 2022

Published in Neural Computation (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2203.08089

Mathematics Subject Classification ID

Mathematical psychology (91E99) Neural networks for/in biological studies, artificial life and related topics (92B20)

Abstract: Barlow (1985) hypothesized that the co-occurrence of two events $A$ and $B$ is "suspicious" if $P(A,B)ggP(A)P(B)$. We first review classical measures of association for $2imes2$ contingency tables, including Yule's $Y$ (Yule, 1912), which depends only on the odds ratio $lambda$, and is independent of the marginal probabilities of the table. We then discuss the mutual information (MI) and pointwise mutual information (PMI), which depend on the ratio $P(A,B)/P(A)P(B)$, as measures of association. We show that, once the effect of the marginals is removed, MI and PMI behave similarly to $Y$ as functions of $lambda$. The pointwise mutual information is used extensively in some research communities for flagging suspicious coincidences, but it is important to bear in mind the sensitivity of the PMI to the marginals, with increased scores for sparser events.

Recommendations

• Coincidences and estimation of entropies of random variables with large cardinalities
• scientific article; zbMATH DE number 885343
• Mutual Information Bounds via Adjacency Events
• Mutual information and the F-theorem
• Correlation detection and an operational interpretation of the Rényi mutual information
• scientific article; zbMATH DE number 1864639
• A NON-UNIFORM BOUND ON THE PROBLEM OF COINCIDENCES
• A new approach to mutual information
• Total correlations and mutual information
• On one extremal problem for mutual information

Cites work

• Frequentist Performance of Bayesian Confidence Intervals for Comparing Proportions in 2 × 2 Contingency Tables
• scientific article; zbMATH DE number 6107964 (Why is no real title available?)
• scientific article; zbMATH DE number 1408945 (Why is no real title available?)

Cited in

(2)

• Statistical inference for natural language processing algorithms with a demonstration using type 2 diabetes prediction from electronic health record notes
• Associative updates for temporal contingencies

This page was built for publication: On Suspicious Coincidences and Pointwise Mutual Information

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5044169)