Infinite Shannon entropy

DOI10.1088/1742-5468/2013/04/P04010MaRDI QIDQ3301575zbMATH OpenOpenAlexWikidataFDO

Authors Valentina Baccetti, Matt Visser

Publication date 11 August 2020

Published in Journal of Statistical Mechanics: Theory and Experiment (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1212.5630

Mathematics Subject Classification ID

Measures of information, entropy (94A17) Classical equilibrium statistical mechanics (general) (82B05)

Abstract: Even if a probability distribution is properly normalizable, its associated Shannon (or von Neumann) entropy can easily be infinite. We carefully analyze conditions under which this phenomenon can occur. Roughly speaking, this happens when arbitrarily small amounts of probability are dispersed into an infinite number of states; we shall quantify this observation and make it precise. We develop several particularly simple, elementary, and useful bounds, and also provide some asymptotic estimates, leading to necessary and sufficient conditions for the occurrence of infinite Shannon entropy. We go to some effort to keep technical computations as simple and conceptually clear as possible. In particular, we shall see that large entropies cannot be localized in state space; large entropies can only be supported on an exponentially large number of states. We are for the time being interested in single-channel Shannon entropy in the information theoretic sense, not entropy in a stochastic field theory or QFT defined over some configuration space, on the grounds that this simple problem is a necessary precursor to understanding infinite entropy in a field theoretic context.

Recommendations

• Entropy of infinite systems and transformations
• Min- and max-entropy in infinite dimensions
• scientific article; zbMATH DE number 503422
• Basics of Shannon entropy
• Some properties of Rényi entropy over countably infinite alphabets

Cites work

• scientific article; zbMATH DE number 3228255 (Why is no real title available?)
• scientific article; zbMATH DE number 3062467 (Why is no real title available?)
• A Mathematical Theory of Communication
• Area laws in quantum systems: mutual information and correlations
• Black holes and entropy
• Colloquium: area laws for the entanglement entropy
• Entropy and area
• Information theory and network coding
• On the quantification of entanglement in infinite-dimensional quantum systems
• Quantum source of entropy for black holes
• Thermodynamics of Spacetime: The Einstein Equation of State
• Unusual quantum states: non–locality, entropy, Maxwell's demon and fractals

Cited in

(6)

• Fundamental limits on quantum dynamics based on entropy change
• Entropy-based test for generalised Gaussian distributions
• Infinite Divisibility of Information
• A note on a Bernstein-type inequality for the log-likelihood function of categorical variables with infinitely many levels
• An optimal uniform concentration inequality for discrete entropies on finite alphabets in the high-dimensional setting
• Entanglement and fermionization of two distinguishable fermions in a strict and non strict one-dimensional space

This page was built for publication: Infinite Shannon entropy

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3301575)