Mean Entropies

DOI10.1007/S11080-005-0926-ZMaRDI QIDQ3368413zbMATH OpenOpenAlexFDO

Authors Bernard H. Lavenda

Publication date 30 January 2006

Published in Open Systems & Information Dynamics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/cond-mat/0404741

zbMATH Keywords

mean code lengthsRényi entropyShannon entropy

Mathematics Subject Classification ID

Measures of information, entropy (94A17)

Abstract: Entropies must correspond to mean values for them to be measurable. The Shannon entropy corresponds to the weighted arithmetic mean, whereas the Renyi entropy corresponds to the exponential mean. These means refer to code lengths, which are converted into entropies by replacing the length of a sequence by the negative logarithm of the probability of its occurrence. Only affine and exponential generating functions of means preserve the property of additivity and invariance under translations, and hence are Kolmogorov-Nagumo functions, resulting in the Shannon and Renyi entropies, respectively. Pseudo-additive entropies are generating functions of means of order 0le au<1, which is the exponential Renyi entropy, or, in the au=0 limit, the Shannon entropy. Means of any order cannot be expressed as escort averages because such averages contradict the fact that the means are monotonically increasing functions of their order. Exponential mean error functions of Renyi, in general, and Shannon, in particular, are shown to be measures of the extent of a distribution.

Recommendations

• On a generalized entropy and a coding theorem
• Four families of measures of entropy
• Some more remarks on a generalized `useful' Havrda-Charvat and Tsallis's entropy
• scientific article; zbMATH DE number 3173999
• On scale and concentration invariance in entropies.

Cites work

• A coding theorem and Rényi's entropy
• Definition of entropy by means of a coding problem
• Exponential entropy as a measure of extent of a distribution
• Generalized information functions
• scientific article; zbMATH DE number 3143967 (Why is no real title available?)
• scientific article; zbMATH DE number 3143969 (Why is no real title available?)
• Inequalities: theory of majorization and its applications
• Information and Coding Discrimination of Pseudo-Additive Entropies (PAE)
• Possible generalization of Boltzmann-Gibbs statistics.

Cited in

(11)

• On scale and concentration invariance in entropies.
• scientific article; zbMATH DE number 3891242 (Why is no real title available?)
• Tsallis entropy: axiomatic characterization and application
• Equivalence of partition functions leads to classification of entropies and means
• Information and Coding Discrimination of Pseudo-Additive Entropies (PAE)
• Averaging entropy
• scientific article; zbMATH DE number 1415687 (Why is no real title available?)
• scientific article; zbMATH DE number 5269043 (Why is no real title available?)
• Contribution of maximum entropy principle in the field of queueing theory
• Fractional Lindley distribution generated by time scale theory, with application to discrete-time lifetime data
• Entropy and effective support size

This page was built for publication: Mean Entropies

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3368413)