Infinite Divisibility of Information

DOI10.1109/TIT.2022.3156432MaRDI QIDQ5088550zbMATH OpenOpenAlexFDO

Authors Cheuk Ting Li

Publication date 13 July 2022

Published in IEEE Transactions on Information Theory (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2008.06092

zbMATH Keywords

information analogue of infinitely divisible probability distributions

Mathematics Subject Classification ID

Probability distributions: general theory (60E05) Measures of information, entropy (94A17)

Abstract: We study an information analogue of infinitely divisible probability distributions, where the i.i.d. sum is replaced by the joint distribution of an i.i.d. sequence. A random variable $X$ is called informationally infinitely divisible if, for any $nge1$, there exists an i.i.d. sequence of random variables $Z_1,ldots,Z_n$ that contains the same information as $X$, i.e., there exists an injective function $f$ such that $X=f(Z_1,ldots,Z_n)$. While there does not exist informationally infinitely divisible discrete random variable, we show that any discrete random variable $X$ has a bounded multiplicative gap to infinite divisibility, that is, if we remove the injectivity requirement on $f$, then there exists i.i.d. $Z_1,ldots,Z_n$ and $f$ satisfying $X=f(Z_1,ldots,Z_n)$, and the entropy satisfies $H(X)/nleH(Z_1)le1.59H(X)/n+2.43$. We also study a new class of discrete probability distributions, called spectral infinitely divisible distributions, where we can remove the multiplicative gap $1.59$. Furthermore, we study the case where $X=(Y_1,ldots,Y_m)$ is itself an i.i.d. sequence, $mge2$, for which the multiplicative gap $1.59$ can be replaced by $1+5sqrt(logm)/m$. This means that as $m$ increases, $(Y_1,ldots,Y_m)$ becomes closer to being spectral infinitely divisible in a uniform manner. This can be regarded as an information analogue of Kolmogorov's uniform theorem. Applications of our result include independent component analysis, distributed storage with a secrecy constraint, and distributed random number generation.

Recommendations

• Infinite Shannon entropy
• Infinite-dimensional divergence information analysis
• scientific article; zbMATH DE number 5046294
• Statistical estimation of the structure of a finite population
• Information-theoretic incompleteness
• Information-Theoretic Incompleteness
• The limits of information
• Finite self-information

Cited in

(1)

• scientific article; zbMATH DE number 5046294 (Why is no real title available?)

This page was built for publication: Infinite Divisibility of Information

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5088550)