Bounds on the Entropy of a Function of a Random Variable and Their Applications

DOI10.1109/TIT.2017.2787181MaRDI QIDQ4569176zbMATH OpenOpenAlexFDO

Authors Ferdinando Cicalese, Luisa Gargano, Ugo Vaccaro

Publication date 27 June 2018

Published in IEEE Transactions on Information Theory (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1712.07906

Mathematics Subject Classification ID

Inequalities; stochastic orderings (60E15) Measures of information, entropy (94A17)

Abstract: It is well known that the entropy $H(X)$ of a discrete random variable $X$ is always greater than or equal to the entropy $H(f(X))$ of a function $f$ of $X$, with equality if and only if $f$ is one-to-one. In this paper, we give tight bounds on $H(f(X))$ when the function $f$ is not one-to-one, and we illustrate a few scenarios where this matters. As an intermediate step towards our main result, we derive a lower bound on the entropy of a probability distribution, when only a bound on the ratio between the maximal and minimal probabilities is known. The lower bound improves on previous results in the literature, and it could find applications outside the present scenario.

Cited in

(9)

• Difference equations related to majorization theorems via Montgomery identity and Green's functions with application to the Shannon entropy
• Entropy and maximizing measures of generic continuous functions
• Bounds for entropy and divergence for distributions over a two-element set
• scientific article; zbMATH DE number 4208747 (Why is no real title available?)
• scientific article; zbMATH DE number 3466395 (Why is no real title available?)
• Stochastic comparisons and dynamic information of random lifetimes in a replacement model
• Efficient optimal Kolmogorov approximation of random variables
• A note on three new bounds of Tsallis entropy
• A universal quantum certainty relation for arbitrary number of observables

This page was built for publication: Bounds on the Entropy of a Function of a Random Variable and Their Applications

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4569176)