Optimal high-dimensional and nonparametric distributed testing under communication constraints

DOI10.1214/23-AOS2269arXiv2202.00968OpenAlexW4386035725MaRDI QIDQ6136575FDOQ6136575

Botond Szabó, Lasse Vuursteen, Harry van Zanten

Publication date: 31 August 2023

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: We derive minimax testing errors in a distributed framework where the data is split over multiple machines and their communication to a central machine is limited to $b$ bits. We investigate both the $d$- and infinite-dimensional signal detection problem under Gaussian white noise. We also derive distributed testing algorithms reaching the theoretical lower bounds. Our results show that distributed testing is subject to fundamentally different phenomena that are not observed in distributed estimation. Among our findings, we show that testing protocols that have access to shared randomness can perform strictly better in some regimes than those that do not. We also observe that consistent nonparametric distributed testing is always possible, even with as little as $1$-bit of communication and the corresponding test outperforms the best local test using only the information available at a single local machine. Furthermore, we also derive adaptive nonparametric distributed testing strategies and the corresponding theoretical lower bounds.

Full work available at URL: https://arxiv.org/abs/2202.00968

zbMATH Keywords

hypothesis testingminimax optimalnonparametricdistributed methods

Mathematics Subject Classification ID

Parametric hypothesis testing (62F03) Nonparametric hypothesis testing (62G10) Parametric inference under constraints (62F30)

Cites Work

• Adaptive hypothesis testing using wavelets
• High-Dimensional Probability
• Hypothesis testing for densities and high-dimensional multinomials: sharp local minimax rates
• Nonparametric goodness-of-fit testing under Gaussian models
• The Brascamp-Lieb inequalities: Finiteness, structure and extremals
• Gaussian kernels have only Gaussian maximizers
• Best constants in Young's inequality, its converse, and its generalization to more than three functions
• Subadditivity of the entropy and its relation to Brascamp-Lieb type inequalities
• Statistical inference under multiterminal data compression
• Decentralized detection by a large number of sensors
• Hypothesis testing with communication constraints
• Communication Complexity
• Communication lower bounds for statistical estimation problems via a distributed data processing inequality
• Adaptive distributed methods under communication constraints
• Distributed adaptive Gaussian mean estimation with unknown variance: interactive protocol helps adaptation
• Distributed nonparametric function estimation: optimal rate of convergence and cost of adaptation
• Inference Under Information Constraints II: Communication Constraints and Shared Randomness
• Distributed function estimation: adaptation using minimal communication
• Optimal Distributed Composite Testing in High-Dimensional Gaussian Models With 1-Bit Communication
• Decentralized Hypothesis Testing in Energy Harvesting Wireless Sensor Networks
• Inference Under Information Constraints I: Lower Bounds From Chi-Square Contraction
• Title not available (Why is that?)
• Information-Theoretic Lower Bounds on Bayes Risk in Decentralized Estimation

Cited In (1)

• Distributed hypothesis testing for large dimensional two-sample mean vectors

This page was built for publication: Optimal high-dimensional and nonparametric distributed testing under communication constraints

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6136575)