Some observations on the concepts of information-theoretic entropy and randomness

DOI10.3390/E3010001MaRDI QIDQ1613002zbMATH OpenOpenAlexFDO

Authors J. D. H. Smith

Publication date 10 September 2002

Published in Entropy (Search for Journal in Brave)

Full work available at URL http://www.mdpi.org/entropy/list01.htm

zbMATH Keywords

information-theoretic entropyKullback entropyrandomness

Mathematics Subject Classification ID

Measures of information, entropy (94A17)

Review text

Summary: Certain aspects of the history, derivation, and physical application of the information-theoretic entropy concept are discussed. Pre-dating Shannon, the concept is traced back to Pauli. A derivation from first principles is given, without use of approximations. The concept depends on the underlying degree of randomness. In physical applications, this translates to dependence on the experimental apparatus available. An example illustrates how this dependence affects Prigogine's proposal for the use of the Second Law of Thermodynamics as a selection principle for the breaking of time symmetry. The dependence also serves to yield a resolution of the so-called ``Gibbs paradox. Extension of the concept from the discrete to the continuous case is discussed. The usual extension is shown to be dimensionally incorrect. Correction introduces a reference density, leading to the concept of Kullback entropy. Practical relativistic considerations suggest a possible proper reference density.

Recommendations

• Differential entropy and time
• Shannon versus Kullback-Leibler entropies in nonequilibrium random motion
• scientific article; zbMATH DE number 3925962
• On the notions of entropy and information
• A farewell to entropy: statistical thermodynamics based on information.

Cited in

(9)

• Differential entropy and dynamics of uncertainty
• 4. On the power of random information
• scientific article; zbMATH DE number 3865143 (Why is no real title available?)
• Entropy, Randomization, Derandomization, and Discrepancy
• scientific article; zbMATH DE number 3255338 (Why is no real title available?)
• Regularities unseen, randomness observed: Levels of entropy convergence
• Some remarks on the definition of Boltzmann, Shannon and Kolmogorov entropy
• Entropy and effective support size
• Shannon versus Kullback-Leibler entropies in nonequilibrium random motion

This page was built for publication: Some observations on the concepts of information-theoretic entropy and randomness

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1613002)