Mutual Information, Relative Entropy and Estimation Error in Semi-Martingale Channels

DOI10.1109/TIT.2018.2829889MaRDI QIDQ4682943zbMATH OpenOpenAlexFDO

Authors Jiantao Jiao, Kartik Venkat, Tsachy Weissman

Publication date 19 September 2018

Published in IEEE Transactions on Information Theory (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1704.05199

Mathematics Subject Classification ID

Measures of information, entropy (94A17) Channel models (including quantum) in information and communication theory (94A40)

Abstract: Fundamental relations between information and estimation have been established in the literature for the continuous-time Gaussian and Poisson channels, in a long line of work starting from the classical representation theorems by Duncan and Kabanov respectively. In this work, we demonstrate that such relations hold for a much larger family of continuous-time channels. We introduce the family of semi-martingale channels where the channel output is a semi-martingale stochastic process, and the channel input modulates the characteristics of the semi-martingale. For these channels, which includes as a special case the continuous time Gaussian and Poisson models, we establish new representations relating the mutual information between the channel input and output to an optimal causal filtering loss, thereby unifying and considerably extending results from the Gaussian and Poisson settings. Extensions to the setting of mismatched estimation are also presented where the relative entropy between the laws governing the output of the channel under two different input distributions is equal to the cumulative difference between the estimation loss incurred by using the mismatched and optimal causal filters respectively. The main tool underlying these results is the Doob--Meyer decomposition of a class of likelihood ratio sub-martingales. The results in this work can be viewed as the continuous-time analogues of recent generalizations for relations between information and estimation for discrete-time L'evy channels.

Recommendations

• Mutual Information, Relative Entropy, and Estimation in the Poisson Channel
• Mutual Information and Minimum Mean-Square Error in Gaussian Channels
• On Mutual Information, Likelihood Ratios, and Estimation Error for the Additive Gaussian Channel
• Some inequalities for channel capacities, mutual informations and mean-squared errors
• Mutual information in stationary channels with additive noise
• Mutual Information and Optimality of Approximate Message-Passing in Random Linear Estimation
• Some relations between mutual information and estimation error in Wiener space
• On mutual information estimation for mixed-pair random variables
• On the effect of receiver estimation error upon channel mutual information

Cited in

(2)

• Some relations between mutual information and estimation error in Wiener space
• Mutual Information and Optimality of Approximate Message-Passing in Random Linear Estimation

This page was built for publication: Mutual Information, Relative Entropy and Estimation Error in Semi-Martingale Channels

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4682943)