Derivatives of Entropy Rate in Special Families of Hidden Markov Chains

DOI10.1109/TIT.2007.899467zbMATH Open1325.94037arXivcs/0603059OpenAlexW2120455086WikidataQ105584790 ScholiaQ105584790MaRDI QIDQ3548932FDOQ3548932

Authors: Guangyue Han, Brian Marcus 

Publication date: 21 December 2008

Published in: IEEE Transactions on Information Theory (Search for Journal in Brave)

Abstract: Consider a hidden Markov chain obtained as the observation process of an ordinary Markov chain corrupted by noise. Zuk, et. al. [13], [14] showed how, in principle, one can explicitly compute the derivatives of the entropy rate of at extreme values of the noise. Namely, they showed that the derivatives of standard upper approximations to the entropy rate actually stabilize at an explicit finite time. We generalize this result to a natural class of hidden Markov chains called ``Black Holes. We also discuss in depth special cases of binary Markov chains observed in binary symmetric noise, and give an abstract formula for the first derivative in terms of a measure on the simplex due to Blackwell.

Full work available at URL: https://arxiv.org/abs/cs/0603059

Recommendations

• Bounds on the entropy rate of binary hidden Markov processes
• Entropy rate for hidden Markov chains with rare transitions
• On the entropy of a hidden Markov process
• Analyticity of Entropy Rate of Hidden Markov Chains
• The capacity of finite-state channels in the high-noise regime

Mathematics Subject Classification ID

Signal theory (characterization, reconstruction, filtering, etc.) (94A12) Markov chains (discrete-time Markov processes on discrete state spaces) (60J10)

Cited In (9)

• Bounds on the entropy rate of binary hidden Markov processes
• Analyticity of Entropy Rate of Hidden Markov Chains
• Asymptotics of input-constrained binary symmetric channel capacity
• On the absolute continuity of the Blackwell measure
• Different closed-form expressions for generalized entropy rates of Markov chains
• Entropy rate for hidden Markov chains with rare transitions
• Stationary measures for projective transformations: the Blackwell and Furstenberg measures
• Shannon entropy rate of hidden Markov processes
• The capacity of finite-state channels in the high-noise regime

This page was built for publication: Derivatives of Entropy Rate in Special Families of Hidden Markov Chains

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3548932)