Optimal testing of discrete distributions with high probability

DOI10.1145/3406325.3450997MaRDI QIDQ6087025zbMATH OpenOpenAlexFDO

Authors Ilias Diakonikolas, Themis Gouleakis, Daniel M. Kane, John Peebles, Eric Price

Publication date 14 November 2023

Published in Proceedings of the 53rd Annual ACM SIGACT Symposium on Theory of Computing (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2009.06540

zbMATH Keywords

distribution property testinghigh confidencesampling

Mathematics Subject Classification ID

Theory of computing (68Qxx)

Abstract: We study the problem of testing discrete distributions with a focus on the high probability regime. Specifically, given samples from one or more discrete distributions, a property $mathcalP$, and parameters $0<epsilon,delta<1$, we want to distinguish {em with probability at least $1-delta$} whether these distributions satisfy $mathcalP$ or are $epsilon$-far from $mathcalP$ in total variation distance. Most prior work in distribution testing studied the constant confidence case (corresponding to $delta=Omega(1)$), and provided sample-optimal testers for a range of properties. While one can always boost the confidence probability of any such tester by black-box amplification, this generic boosting method typically leads to sub-optimal sample bounds. Here we study the following broad question: For a given property $mathcalP$, can we {em characterize} the sample complexity of testing $mathcalP$ as a function of all relevant problem parameters, including the error probability $delta$? Prior to this work, uniformity testing was the only statistical task whose sample complexity had been characterized in this setting. As our main results, we provide the first algorithms for closeness and independence testing that are sample-optimal, within constant factors, as a function of all relevant parameters. We also show matching information-theoretic lower bounds on the sample complexity of these problems. Our techniques naturally extend to give optimal testers for related problems. To illustrate the generality of our methods, we give optimal algorithms for testing collections of distributions and testing closeness with unequal sized samples.

Cited in

(7)

• scientific article; zbMATH DE number 3864309 (Why is no real title available?)
• Optimal sampling schemes for the Gaussian hypothesis testing problem
• Collision-based Testers are Optimal for Uniformity and Closeness
• The Benjamini-Hochberg Method in the Case of Discrete Test Statistics
• Topics and Techniques in Distribution Testing: A Biased but Representative Sample
• Private distribution testing with heterogeneous constraints: your epsilon might not be mine
• Complexity of high-dimensional identity testing with coordinate conditional sampling

This page was built for publication: Optimal testing of discrete distributions with high probability

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6087025)