Information Theoretic Proofs of Entropy Power Inequalities

DOI10.1109/TIT.2010.2090193MaRDI QIDQ5281108zbMATHOpenAlexFDO

Authors Olivier Rioul

Publication date 27 July 2017

Published in IEEE Transactions on Information Theory (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/0704.1751

Mathematics Subject Classification ID

Information theory (general) (94A15) Measures of information, entropy (94A17)

Abstract: While most useful information theoretic inequalities can be deduced from the basic properties of entropy or mutual information, up to now Shannon's entropy power inequality (EPI) is an exception: Existing information theoretic proofs of the EPI hinge on representations of differential entropy using either Fisher information or minimum mean-square error (MMSE), which are derived from de Bruijn's identity. In this paper, we first present an unified view of these proofs, showing that they share two essential ingredients: 1) a data processing argument applied to a covariance-preserving linear transformation; 2) an integration over a path of a continuous Gaussian perturbation. Using these ingredients, we develop a new and brief proof of the EPI through a mutual information inequality, which replaces Stam and Blachman's Fisher information inequality (FII) and an inequality for MMSE by Guo, Shamai and Verd'u used in earlier proofs. The result has the advantage of being very simple in that it relies only on the basic properties of mutual information. These ideas are then generalized to various extended versions of the EPI: Zamir and Feder's generalized EPI for linear transformations of the random variables, Takano and Johnson's EPI for dependent variables, Liu and Viswanath's covariance-constrained EPI, and Costa's concavity inequality for the entropy power.

Cited in

(20)

• An extension of entropy power inequality for dependent random variables
• Clock synchronization and estimation in highly dynamic networks: an information theoretic approach
• Conditional quantum entropy power inequality ford-level quantum systems
• Weighted \(p\)-Rényi entropy power inequality: information theory to quantum Shannon theory
• 4. On the power of random information
• A primer on alpha-information theory with application to leakage in secrecy systems
• Extension of de Bruijn's identity to dependent non-Gaussian noise channels
• A simple proof of the entropy-power inequality
• Information theoretic inequalities
• A concavity property for the reciprocal of Fisher information and its consequences on Costa's EPI
• Entropy power inequalities for qudits
• Weighted entropy: basic inequalities
• Role of information theoretic uncertainty relations in quantum theory
• Asymptotic form of the Kullback-Leibler divergence for multivariate asymmetric heavy-tailed distributions
• On the smoothed minimum error entropy criterion
• An information - theoretic proof of Hadamard's inequality (Corresp.)
• The information-theoretic meaning of Gagliardo-Nirenberg type inequalities
• On Shannon's formula and Hartley's rule: beyond the mathematical coincidence
• A de Bruijn's identity for dependent random variables based on copula theory
• Quantum Rényi entropy functionals for bosonic Gaussian systems

This page was built for publication: Information Theoretic Proofs of Entropy Power Inequalities

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5281108)