Information Theoretic Proofs of Entropy Power Inequalities

DOI10.1109/TIT.2010.2090193MaRDI QIDQ5281108zbMATH OpenOpenAlexFDO

Authors Olivier Rioul

Publication date 27 July 2017

Published in IEEE Transactions on Information Theory (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/0704.1751

Mathematics Subject Classification ID

Information theory (general) (94A15) Measures of information, entropy (94A17)

Abstract: While most useful information theoretic inequalities can be deduced from the basic properties of entropy or mutual information, up to now Shannon's entropy power inequality (EPI) is an exception: Existing information theoretic proofs of the EPI hinge on representations of differential entropy using either Fisher information or minimum mean-square error (MMSE), which are derived from de Bruijn's identity. In this paper, we first present an unified view of these proofs, showing that they share two essential ingredients: 1) a data processing argument applied to a covariance-preserving linear transformation; 2) an integration over a path of a continuous Gaussian perturbation. Using these ingredients, we develop a new and brief proof of the EPI through a mutual information inequality, which replaces Stam and Blachman's Fisher information inequality (FII) and an inequality for MMSE by Guo, Shamai and Verd'u used in earlier proofs. The result has the advantage of being very simple in that it relies only on the basic properties of mutual information. These ideas are then generalized to various extended versions of the EPI: Zamir and Feder's generalized EPI for linear transformations of the random variables, Takano and Johnson's EPI for dependent variables, Liu and Viswanath's covariance-constrained EPI, and Costa's concavity inequality for the entropy power.

Cited in

(21)

• A concavity property for the reciprocal of Fisher information and its consequences on Costa's EPI
• Weighted entropy: basic inequalities
• Asymptotic form of the Kullback-Leibler divergence for multivariate asymmetric heavy-tailed distributions
• A primer on alpha-information theory with application to leakage in secrecy systems
• The information-theoretic meaning of Gagliardo-Nirenberg type inequalities
• Entropy power inequalities for qudits
• On Shannon's formula and Hartley's rule: beyond the mathematical coincidence
• Role of information theoretic uncertainty relations in quantum theory
• Extension of de Bruijn's identity to dependent non-Gaussian noise channels
• 4. On the power of random information
• An information - theoretic proof of Hadamard's inequality (Corresp.)
• Clock synchronization and estimation in highly dynamic networks: an information theoretic approach
• A simple proof of the entropy-power inequality
• Information theoretic inequalities
• On the smoothed minimum error entropy criterion
• Conditional quantum entropy power inequality ford-level quantum systems
• An extension of entropy power inequality for dependent random variables
• A de Bruijn's identity for dependent random variables based on copula theory
• Weighted \(p\)-Rényi entropy power inequality: information theory to quantum Shannon theory
• Quantum Rényi entropy functionals for bosonic Gaussian systems
• Title not available (Why is no real title available?)

This page was built for publication: Information Theoretic Proofs of Entropy Power Inequalities

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5281108)