Multivariate Information Bottleneck

DOI10.1162/NECO.2006.18.8.1739MaRDI QIDQ5423492zbMATH OpenOpenAlexDBLPWikidataFDO

Authors Noam Slonim, Nir Friedman, Naftali Tishby

Publication date 23 October 2007

Published in Neural Computation (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1301.2270

Mathematics Subject Classification ID

Data structures (68P05) Pattern recognition, speech recognition (68T10)

Abstract: The Information bottleneck method is an unsupervised non-parametric data organization technique. Given a joint distribution P(A,B), this method constructs a new variable T that extracts partitions, or clusters, over the values of A that are informative about B. The information bottleneck has already been applied to document classification, gene expression, neural code, and spectral analysis. In this paper, we introduce a general principled framework for multivariate extensions of the information bottleneck method. This allows us to consider multiple systems of data partitions that are inter-related. Our approach utilizes Bayesian networks for specifying the systems of clusters and what information each captures. We show that this construction provides insight about bottleneck variations and enables us to characterize solutions of these variations. We also present a general framework for iterative algorithms for constructing solutions, and apply it to several examples.

Recommendations

• The information bottleneck and geometric clustering
• Learning and Generalization with the Information Bottleneck
• Learning and generalization with the information bottleneck
• The deterministic information bottleneck
• scientific article; zbMATH DE number 5957257

Cites work

• A Mathematical Theory of Communication
• Asymptotically optimal classification for multiple tests with empirically observed statistics

Cited in

(19)

• Pursuit of food \textit{versus} pursuit of information in a Markovian perception-action loop model of foraging
• A framework to uncover multiple alternative clusterings
• Hierarchical clustering based on the information bottleneck method using a control process
• Robust classifying of prokaryotic genomes
• Annealing and the normalized N-cut
• scientific article; zbMATH DE number 5957257 (Why is no real title available?)
• Learning and Generalization with the Information Bottleneck
• The mathematical structure of information bottleneck methods
• Applying the information bottleneck to statistical relational learning
• scientific article; zbMATH DE number 1945220 (Why is no real title available?)
• INFORMATION BOTTLENECKS, CAUSAL STATES, AND STATISTICAL RELEVANCE BASES: HOW TO REPRESENT RELEVANT INFORMATION IN MEMORYLESS TRANSDUCTION
• Gaussian lower bound for the information bottleneck limit
• The information bottleneck and geometric clustering
• The deterministic information bottleneck
• Predictive Coding and the Slowness Principle: An Information-Theoretic Approach
• Factorized mutual information maximization.
• Semi-supervised model-based document clustering: a comparative study
• Learning decision trees with taxonomy of propositionalized attributes
• Learning and generalization with the information bottleneck

This page was built for publication: Multivariate Information Bottleneck

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5423492)