Linear dependence structure of the entropy space
The author studies linear dependence relations among Shannon's and McGill's information as well as the lattice-theoretic description of them to construct a possible mathematical foundation for information-theoretic multivariate analysis. Let \(\underline X\) be the Boolean lattice formed of marginal random vectors of an \(n\)-dimensional random variable \(X^n = (X_1,\ldots,X_n)\), and let \(\overline H(\underline X)\) (entropy space) be the vector space generated by all the entropies \(h(\alpha)\) \((\alpha\in\underline X)\). Then, the main results are summarized as follows: 1) The difference \(\Delta h(\alpha) = \displaystyle\sum_{\gamma\le \alpha} \mu(\gamma,\alpha)h(\gamma)\) \((\mu(\gamma,\alpha)\): Möbius function of \(\underline X\)) coincides with McGill's multiple mutual information. Similarly for the conditional difference \(\Delta_\sigma h(\alpha)\). 2) The subspace \(\overline H_0(\underline X)\) (correlative entropy space) of \(\overline H(\underline X)\) is defined as the set of all the elements in \(\overline H(\underline X)\) the values of which vanish for independent distributions. Such elements may be regarded as information-theoretic measures of multivariate analysis. 3) The set \(\underline H_1 = \{\Delta h(\alpha\mid r(\alpha)\ge 1\}\) \((r(\alpha)\): rank of \(\alpha)\) is a basis of \(\overline H(\underline X)\), whereas the set \(\underline H_2) = \{\Delta h(\alpha\mid r(\alpha)\ge 2\}\) is a basis of \(\overline H_0(\underline X)\). The dimension of \(\overline H(\underline X)\) is \(2^n - 1\), and that of \(\overline H_0(\underline X)\) is \(2^n - (n +1)\). 4) \(\Delta_\sigma h(\alpha)\) is expanded in terms of \(\Delta_\sigma h(\beta)\) as follows: \[ \Delta_\sigma h(\alpha) = \sum_{\gamma\le\sigma} \Delta h((\alpha\wedge\bar\sigma)\vee \gamma)\quad (\bar\sigma: \text{ complement of }\sigma). \] 5) There exists a probability distribution under which the values of all the elements of \(\underline H_2\) coincide with arbitrarily prescribed nonpositive values. 6) Any linear dependence relations among Shannon's (conditional) mutual information is generated by a particular type of linear dependence relations, i.e., those among chain-forming information on 3-dimensional sublattices. 7) It is shown that the above results are valid, with obvious modifications, for the dual lattice \(\underline X^*\) of \(\underline X\). In parallel with Watanabe's total correlation \(g(X^n) = \sum_{i=1}^n h(X_i) - h(X^n)\), the author defines the dual total correlation \[ g^*(\emptyset) = \sum_{i=1}^n h(\overline X_i) - (n-1)h(X^n), \] which is also nonnegative and vanishes if and only if \(X_1,\ldots, X_n\) are independent. As is well known, this property is a basis for multivariate-analytic interpretation of Watanabe's total correlation.
- On the entropy of linear factors
- Dimensionally sharp inequalities for the linear entropy
- Relative Entropy Measures of Multivariate Dependence
- On generalized measures of entropy and dependence
- scientific article; zbMATH DE number 2014348
- Linear response theory of entanglement entropy
- On the entropic measures of stochastic dependence
- scientific article; zbMATH DE number 3925877
- Entropies on independent occupation spaces
- Synergy, redundancy, and multivariate information measures: an experimentalist's perspective
- Asymptotic dependency structure of multiple signals
- Measure concentration and the weak Pinsker property
- The structure of low-complexity Gibbs measures on product spaces
- On entropy, information inequalities, and groups
- Higher-Order Description of Brain Function
- Multi-variate correlation and mixtures of product measures.
- Squashed entanglement and approximate private states
- On the entropy of linear factors
- Stabilizer information inequalities from phase space distributions
- The Brin Prize works of Tim Austin
- Family of quantum mutual information in multiparty quantum systems
- Information theory for complex systems scientists: what, why, and how
- Multi-party set disjointness and intersection with bounded dependence
- A hitchhiker's guide to information theoretical measures in psychology
- Information processing in linear vector space
This page was built for publication: Linear dependence structure of the entropy space
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4082682)