Entropy estimation in Turing's perspective

DOI10.1162/NECO_A_00266MaRDI QIDQ2919410zbMATH OpenOpenAlexWikidataFDO

Authors Zhiyi Zhang

Publication date 2 October 2012

Published in Neural Computation (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1162/neco_a_00266

Mathematics Subject Classification ID

Statistical aspects of information-theoretic topics (62B10) Measures of information, entropy (94A17)

Review text

In this paper, the author studies the entropy estimator \({\hat H}_z\) which is defined by the arithmetic mean on \(\nu\) for \(\sum_k p_k(1-p_k)^{\nu}\). As main results, a few lemmas and theorems are shown.NEWLINENEWLINE In Lemma 1, the author gives a necessary and sufficient condition for the entropy \(H\) to be finite. In Lemma 2 he gives an upper bound for the variance of the entropy estimator \({\hat H}_z\), and then shows that the entropy estimator \({\hat H}_z\) is a consistent estimator of the entropy \(H\) under the condition that \(H\) is finite (Theorem 1). Using Lemma 2, the author obtains another important theorem, Theorem 2. Examples and remarks are given. Finally, computational results are given and compared with those of other estimators.

Recommendations

• A note on entropy estimation
• On the entropy estimators
• Entropy estimators with almost sure convergence and an \(\mathcal O(n^{-1})\) variance
• On the estimation of entropy
• scientific article; zbMATH DE number 17209

Cites work

• A Class of Statistics with Asymptotically Normal Distribution
• A normal limit law for a nonparametric estimator of the coverage of a random sample
• A sufficient normality condition for Turing's formula
• Asymptotic normality of a nonparametric estimator of sample coverage
• Bias analysis in entropy estimation
• Convergence properties of functional estimates for discrete distributions
• Estimating the Total Probability of the Unobserved Outcomes of an Experiment
• Estimation of Entropy and Mutual Information
• Measurement of Diversity
• Re-parameterization of multinomial distributions and diversity indices
• THE POPULATION FREQUENCIES OF SPECIES AND THE ESTIMATION OF POPULATION PARAMETERS

Cited in

(18)

• A multivariate normal law for Turing's formulae
• An asymptotic lower bound for the entropy of discrete populations with application to the estimation of entropy for approximately uniform populations
• Entropic representation and estimation of diversity indices
• Statistical implications of Turing's formula
• Always Good Turing: asymptotically optimal probability estimation
• Estimating Entropy on<tex>$m$</tex>Bins Given Fewer Than<tex>$m$</tex>Samples
• scientific article; zbMATH DE number 4007449 (Why is no real title available?)
• An Entropy Estimator for a Class of Infinite Alphabet Processes
• A note on entropy estimation
• Nonparametric estimation of Kullback-Leibler divergence
• Existence of an unbiased consistent entropy estimator for the special Bernoulli measure
• On Sub-Gaussian Concentration of Missing Mass
• Entropy-based assessment of biodiversity, with application to ants' nests data
• Turing-entropic tail classification: a nonparametric approach to tail inference
• Bias adjustment for a nonparametric entropy estimator
• A mutual information estimator with exponentially decaying bias
• Entropy estimators with almost sure convergence and an \(\mathcal O(n^{-1})\) variance
• Erratum to: Entropy of Turing machines with moving head

This page was built for publication: Entropy estimation in Turing's perspective

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2919410)