A note on entropy estimation

DOI10.1162/NECO_A_00775MaRDI QIDQ5380327zbMATH OpenOpenAlexWikidataFDO

Authors Thomas Schürmann

Publication date 4 June 2019

Published in Neural Computation (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1503.05911

Mathematics Subject Classification ID

Measures of information, entropy (94A17)

Abstract: We compare an entropy estimator $H_z$ recently discussed in [10] with two estimators $H_1$ and $H_2$ introduced in [6][7]. We prove the identity $H_{z}equiv{H}_1$, which has not been taken into account in [10]. Then, we prove that the statistical bias of $H_1$ is less than the bias of the ordinary likelihood estimator of entropy. Finally, by numerical simulation we verify that for the most interesting regime of small sample estimation and large event spaces, the estimator $H_2$ has a significant smaller statistical error than $H_z$.

Recommendations

• Entropy estimation in Turing's perspective
• Entropy estimators‐improvements and comparisons
• Estimating the entropy of binary time series: methodology, some theory and a simulation study
• Two measures of sample entropy
• A new estimator for Shannon entropy

Cites work

• scientific article; zbMATH DE number 4062892 (Why is no real title available?)
• scientific article; zbMATH DE number 3568534 (Why is no real title available?)
• scientific article; zbMATH DE number 3273551 (Why is no real title available?)
• Bayesian entropy estimation for countable discrete distributions
• Bias analysis in entropy estimation
• Convergence properties of functional estimates for discrete distributions
• Entropy estimation in Turing's perspective
• Entropy estimation of symbol sequences
• Measurement of Diversity
• Prediction and Entropy of Printed English

Cited in

(10)

• Entropy estimation in Turing's perspective
• scientific article; zbMATH DE number 3954044 (Why is no real title available?)
• Estimating the entropy of binary time series: methodology, some theory and a simulation study
• Entropy estimates of small data sets
• An asymptotic lower bound for the entropy of discrete populations with application to the estimation of entropy for approximately uniform populations
• A note on entropy optimization
• Estimating entropy rate from censored symbolic time series: A test for time-irreversibility
• A proof of the estimation from below in Pesin's entropy formula
• Symbolic partition in chaotic maps
• Scaling behaviour of entropy estimates

This page was built for publication: A note on entropy estimation

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5380327)