Relative entropy via non-sequential recursive pair substitution

DOI10.1088/1742-5468/2010/09/P09010MaRDI QIDQ3301194zbMATH OpenOpenAlexFDO

Authors

Publication date 11 August 2020

Published in Journal of Statistical Mechanics: Theory and Experiment (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1007.3384

Mathematics Subject Classification ID

Measures of information, entropy (94A17)

Abstract: The entropy of an ergodic source is the limit of properly rescaled 1-block entropies of sources obtained applying successive non-sequential recursive pairs substitutions (see P. Grassberger 2002 ArXiv:physics/0207023 and D. Benedetto, E. Caglioti and D. Gabrielli 2006 Jour. Stat. Mech. Theo. Exp. 09 doi:10.1088/1742.-5468/2006/09/P09011). In this paper we prove that the cross entropy and the Kullback-Leibler divergence can be obtained in a similar way.

Recommendations

• Non-sequential recursive pair substitutions and numerical entropy estimates in symbolic dynamical systems
• A measure of relative entropy between individual sequences with application to universal classification
• Non-sequential recursive pair substitution: some rigorous results
• Renyi's divergence and entropy rates for finite alphabet Markov sources
• On the Entropy Rate of Pattern Processes

Cites work

• A measure of relative entropy between individual sequences with application to universal classification
• A universal algorithm for sequential data compression
• An example of mathematical authorship attribution
• Asymptotic recurrence and waiting times for stationary processes
• Data compression and learning in time sequences analysis
• Entropy and data compression schemes
• scientific article; zbMATH DE number 918233 (Why is no real title available?)
• Non-sequential recursive pair substitution: some rigorous results
• The Similarity Metric
• Universal Divergence Estimation for Finite-Alphabet Sources
• Universal Entropy Estimation Via Block Sorting

Cited in

(4)

• Non-sequential recursive pair substitution: some rigorous results
• Non-sequential recursive pair substitutions and numerical entropy estimates in symbolic dynamical systems
• On the entropy of a family of random substitutions
• Relative entropy tuples, relative u.p.e. and c.p.e. extensions

This page was built for publication: Relative entropy via non-sequential recursive pair substitution

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3301194)