Higher criticism to compare two large frequency tables, with sensitivity to possible rare and weak differences

DOI10.1214/21-AOS2158MaRDI QIDQ2148983zbMATH OpenFDO

Authors David Donoho, Alon Kipnis

Publication date 24 June 2022

Published in The Annals of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2007.01958

zbMATH Keywords

higher criticismtwo-sample testsparse mixturetesting multinomials

Mathematics Subject Classification ID

Nonparametric hypothesis testing (62G10) Hypothesis testing in multivariate analysis (62H15) Contingency tables (62H17)

Recommendations

• Beyond HC: more sensitive tests for rare/weak alternatives
• Higher criticism for large-scale inference, especially for rare and weak effects
• Higher criticism for detecting sparse heterogeneous mixtures.
• The intermediates take it all: asymptotics of higher criticism statistics and a powerful alternative based on equal local levels
• Higher criticism: \(p\)-values and criticism

Cites work

• scientific article; zbMATH DE number 3945130 (Why is no real title available?)
• scientific article; zbMATH DE number 1158743 (Why is no real title available?)
• scientific article; zbMATH DE number 3285848 (Why is no real title available?)
• Collision-based Testers are Optimal for Uniformity and Closeness
• Concentration inequalities. A nonasymptotic theory of independence
• Detection boundary in sparse regression
• Empirical Processes with Applications to Statistics
• Feature selection by higher criticism thresholding achieves the optimal phase diagram
• Global power functions of goodness of fit tests.
• Global testing under sparse alternatives: ANOVA, multiple comparisons and the higher criticism
• Goodness-of-fit tests via phi-divergences
• Higher criticism for detecting sparse heterogeneous mixtures.
• Higher criticism for discriminating word-frequency tables and authorship attribution
• Higher criticism for large-scale inference, especially for rare and weak effects
• Higher criticism thresholding: Optimal feature selection when useful features are rare and weak
• Hypothesis testing for high-dimensional sparse binary regression
• Inference in an Authorship Problem
• On the Lambert \(w\) function
• Optimal Detection of Sparse Mixtures Against a Given Null Distribution
• Optimal algorithms for testing closeness of discrete distributions
• Optimal detection of heterogeneous and heteroscedastic mixtures
• Probability and computing. Randomization and probabilistic techniques in algorithms and data analysis
• THE TRANSFORMATION OF POISSON, BINOMIAL AND NEGATIVE-BINOMIAL DATA
• The collected works of Wassily Hoeffding. Ed. by N. I. Fisher and P. K. Sen
• The intermediates take it all: asymptotics of higher criticism statistics and a powerful alternative based on equal local levels
• The sparse Poisson means model
• Transformation theory: How normal is a family of distributions?

Cited in

(5)

• The impossibility region for detecting sparse mixtures using the higher criticism
• Higher Criticism to Compare Two Large Frequency Tables, with sensitivity to Possible Rare and Weak Differences
• Higher criticism for discriminating word-frequency tables and authorship attribution
• Statistical limits of sparse mixture detection
• Sparse signal detection in heteroscedastic Gaussian sequence models: sharp minimax rates

This page was built for publication: Higher criticism to compare two large frequency tables, with sensitivity to possible rare and weak differences

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2148983)