Summary: Notion of Effective size of support (Ess) of a random variable is introduced. A small set of natural requirements that a measure of Ess should satisfy is presented. The measure with prescribed properties is in a direct (exp-) relationship to the family of Rényi's - entropies which includes also Shannon's entropy \(H\). Considerations of choice of the value of imply that exp(\(H\)) appears to be the most appropriate measure of Ess. Entropy and Ess can be viewed thanks to their log/exp relationship as two aspects of the same thing. In Probability and Statistics the Ess aspect could appear more basic than the entropic one.
- A Mathematical Theory of Communication
- scientific article; zbMATH DE number 3143967 (Why is no real title available?)
- Information Theory and Statistical Mechanics
- On the entropy of continuous probability distributions (Corresp.)
- Rényi information, loglikelihood and an intrinsic distribution measure
- Some observations on the concepts of information-theoretic entropy and randomness
This page was built for publication: Entropy and effective support size
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q925758)
