Lower Bounds for Learning Distributions under Communication Constraints via Fisher Information
From MaRDI portal
Publication:6313775
zbMATH Open1531.68073arXiv1902.02890MaRDI QIDQ6313775FDOQ6313775
Authors: Leighton Pate Barnes, Yanjun Han, Ayfer Özgür 
Publication date: 7 February 2019
Abstract: We consider the problem of learning high-dimensional, nonparametric and structured (e.g. Gaussian) distributions in distributed networks, where each node in the network observes an independent sample from the underlying distribution and can use bits to communicate its sample to a central processor. We consider three different models for communication. Under the independent model, each node communicates its sample to a central processor by independently encoding it into bits. Under the more general sequential or blackboard communication models, nodes can share information interactively but each node is restricted to write at most bits on the final transcript. We characterize the impact of the communication constraint on the minimax risk of estimating the underlying distribution under loss. We develop minimax lower bounds that apply in a unified way to many common statistical models and reveal that the impact of the communication constraint can be qualitatively different depending on the tail behavior of the score function associated with each model. A key ingredient in our proofs is a geometric characterization of Fisher information from quantized samples.
Statistical aspects of information-theoretic topics (62B10) Density estimation (62G07) Learning and adaptive systems in artificial intelligence (68T05) Network design and communication in computer systems (68M10)
This page was built for publication: Lower Bounds for Learning Distributions under Communication Constraints via Fisher Information
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6313775)
