Distributed information-theoretic clustering

DOI10.1093/IMAIAI/IAAB007MaRDI QIDQ5073887zbMATH OpenOpenAlexFDO

Authors Georg Pichler, Pablo Piantanida, Gerald Matz

Publication date 4 May 2022

Published in Information and Inference: A Journal of the IMA (Search for Journal in Brave)

Copyright license Creative Commons Attribution 4.0 International

Full work available at URL https://arxiv.org/abs/1602.04605

zbMATH Keywords

CEO probleminformation bottleneckmutual informationsource coding

Mathematics Subject Classification ID

Classification and discrimination; cluster analysis (statistical aspects) (62H30) Information theory (general) (94A15)

Abstract: We study a novel multi-terminal source coding setup motivated by the biclustering problem. Two separate encoders observe two i.i.d. sequences $X^n$ and $Y^n$, respectively. The goal is to find rate-limited encodings $f(x^n)$ and $g(z^n)$ that maximize the mutual information $I(f(X^n);g(Y^n))/n$. We discuss connections of this problem with hypothesis testing against independence, pattern recognition, and the information bottleneck method. Improving previous cardinality bounds for the inner and outer bounds allows us to thoroughly study the special case of a binary symmetric source and to quantify the gap between the inner and the outer bound in this special case. Furthermore, we investigate a multiple description (MD) extension of the Chief Operating Officer (CEO) problem with mutual information constraint. Surprisingly, this MD-CEO problem permits a tight single-letter characterization of the achievable region.

Recommendations

• Multiterminal source encoding with one distortion criterion
• An improved outer bound for multiterminal source coding
• Generalized Coset Codes for Distributed Binning
• Information-based clustering
• The rate-distortion function for the quadratic Gaussian CEO problem

Cited in

(2)

• Generalized Coset Codes for Distributed Binning
• Information theoretic hierarchical clustering

This page was built for publication: Distributed information-theoretic clustering

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5073887)