Variability as a better characterization of Shannon entropy

DOI10.1088/1361-6404/ABE361MaRDI QIDQ6108033zbMATH OpenOpenAlexFDO

Authors Gabriele Carcassi, Christine A. Aidala, Julian Barbour

Publication date 29 June 2023

Published in European Journal of Physics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1912.02012

zbMATH Keywords

information theoryShannon entropystatistical mechanicsvariability

Mathematics Subject Classification ID

Information theory (general) (94A15) Measures of information, entropy (94A17)

Abstract: The Shannon entropy, one of the cornerstones of information theory, is widely used in physics, particularly in statistical mechanics. Yet its characterization and connection to physics remain vague, leaving ample room for misconceptions and misunderstanding. We will show that the Shannon entropy can be fully understood as measuring the variability of the elements within a given distribution: it characterizes how much variation can be found within a collection of objects. We will see that it is the only indicator that is continuous and linear, that it quantifies the number of yes/no questions (i.e. bits) that are needed to identify an element within the distribution, and we will see how applying this concept to statistical mechanics in different ways leads to the Boltzmann, Gibbs and von Neumann entropies.

Recommendations

• Basics of Shannon entropy
• Shannon entropy: a rigorous notion at the crossroads between probability, information theory, dynamical systems and statistical physics
• Shannon entropy: axiomatic characterization and application
• Probability distribution and entropy as a measure of uncertainty
• The entropy of a discrete real variable

Cites work

• A formal derivation of the Gibbs entropy for classical systems following the Schrödinger quantum mechanical approach
• A Mathematical Theory of Communication
• A Method for the Construction of Minimum-Redundancy Codes
• Gibbs vs Boltzmann Entropies
• scientific article; zbMATH DE number 3143967 (Why is no real title available?)
• scientific article; zbMATH DE number 5665286 (Why is no real title available?)
• scientific article; zbMATH DE number 802685 (Why is no real title available?)
• Information Theory and Statistical Mechanics
• The different paths to entropy
• Time evolution of entropy, in various scenarios
• Time-reversal and entropy

Cited in

(10)

• Effect of class-interval size on entropy
• Advancing Shannon entropy for measuring diversity in systems
• Shannon's entropy as an index of product variety
• Biological models: measuring variability with classical and quantum information
• Basics of Shannon entropy
• scientific article; zbMATH DE number 2134549 (Why is no real title available?)
• The entropy of a discrete real variable
• CAN WE MEASURE THE ORGANIZATIONAL PROPERTIES OF A SYSTEM BY ITS ENTROPIC VARIANCE?
• On the reality of the quantum state once again: a no-go theorem for -ontic models
• Differential Shannon and Rényi entropies revisited

This page was built for publication: Variability as a better characterization of Shannon entropy

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6108033)