An integral representation of the relative entropy

DOI10.3390/E14081469MaRDI QIDQ406130zbMATH OpenOpenAlexFDO

Authors Miku Hirata, Aya Nemoto, Hiroaki Yoshida

Publication date 8 September 2014

Published in Entropy (Search for Journal in Brave)

Full work available at URL https://doi.org/10.3390/e14081469

zbMATH Keywords

de Bruijn identitylogarithmic Sobolev inequalityrelative entropyrelative Fisher informationStam inequality

Mathematics Subject Classification ID

Statistical aspects of information-theoretic topics (62B10) Measures of information, entropy (94A17)

Review text

Summary: Recently the identity of de Bruijn type between the relative entropy and the relative Fisher information with the reference moving has been unveiled by Verdú via MMSE in estimation theory. In this paper, we shall give another proof of this identity in more direct way that the derivative is calculated by applying integrations by part with the heat equation. We shall also derive an integral representation of the relative entropy, as one of the applications of which the logarithmic Sobolev inequality for centered Gaussian measures will be given.

Recommendations

• Representation formula for the entropy and functional inequalities
• Variational representations related to Tsallis relative entropy
• scientific article; zbMATH DE number 7653414
• Integral representations and asymptotic expansions for Shannon and Renyi entropies
• Remarks on the free relative entropy and the free Fisher information distance

Cites work

• A short proof of the "concavity of entropy power"
• Cercignani's conjecture is sometimes true and always almost true
• Entropy and the central limit theorem
• Fisher information inequalities and the central limit theorem
• Generalized Entropy Power Inequalities and Monotonicity Properties of Information
• scientific article; zbMATH DE number 2131215 (Why is no real title available?)
• Information theoretic inequalities
• Logarithmic Sobolev Inequalities
• Mismatched Estimation and Relative Entropy
• Mutual Information and Minimum Mean-Square Error in Gaussian Channels
• ON CONVEX SOBOLEV INEQUALITIES AND THE RATE OF CONVERGENCE TO EQUILIBRIUM FOR FOKKER-PLANCK TYPE EQUATIONS
• Some inequalities satisfied by the quantities of information of Fisher and Shannon
• Superadditivity of Fisher's information and logarithmic Sobolev inequalities
• The convolution inequality for entropy powers

Cited in

(11)

• Relative entropy equals bulk relative entropy
• Affinity of the Arov entropy
• Hellmann-Feynman connection for the relative Fisher information
• A characterization of the relative entropies
• scientific article; zbMATH DE number 431873 (Why is no real title available?)
• Telescopic Relative Entropy
• Representation formula for the entropy and functional inequalities
• scientific article; zbMATH DE number 1136341 (Why is no real title available?)
• scientific article; zbMATH DE number 7653414 (Why is no real title available?)
• Entropy pair realization
• Legendre transform structure and extremal properties of the relative Fisher information

This page was built for publication: An integral representation of the relative entropy

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q406130)