Entropy and effective support size (Q925758): Difference between revisions

Summary: Notion of Effective size of support (Ess) of a random variable is introduced. A small set of natural requirements that a measure of Ess should satisfy is presented. The measure with prescribed properties is in a direct (exp-) relationship to the family of Rényi's - entropies which includes also Shannon's entropy \(H\). Considerations of choice of the value of imply that exp(\(H\)) appears to be the most appropriate measure of Ess. Entropy and Ess can be viewed thanks to their log/exp relationship as two aspects of the same thing. In Probability and Statistics the Ess aspect could appear more basic than the entropic one.