A note on entropy estimation

DOI10.1162/NECO_A_00775zbMATH Open1473.94030arXiv1503.05911OpenAlexW1819409771WikidataQ86016711 ScholiaQ86016711MaRDI QIDQ5380327FDOQ5380327

Authors: Thomas Schürmann 

Publication date: 4 June 2019

Published in: Neural Computation (Search for Journal in Brave)

Abstract: We compare an entropy estimator $H_z$ recently discussed in [10] with two estimators $H_1$ and $H_2$ introduced in [6][7]. We prove the identity $H_{z}equiv{H}_1$, which has not been taken into account in [10]. Then, we prove that the statistical bias of $H_1$ is less than the bias of the ordinary likelihood estimator of entropy. Finally, by numerical simulation we verify that for the most interesting regime of small sample estimation and large event spaces, the estimator $H_2$ has a significant smaller statistical error than $H_z$.

Full work available at URL: https://arxiv.org/abs/1503.05911

Recommendations

• Entropy estimation in Turing's perspective
• Entropy estimators‐improvements and comparisons
• Estimating the entropy of binary time series: methodology, some theory and a simulation study
• Two measures of sample entropy
• A new estimator for Shannon entropy

Mathematics Subject Classification ID

Measures of information, entropy (94A17)

Cites Work

• Title not available (Why is that?)
• Title not available (Why is that?)
• Prediction and Entropy of Printed English
• Measurement of Diversity
• Convergence properties of functional estimates for discrete distributions
• Entropy estimation of symbol sequences
• Bias analysis in entropy estimation
• Title not available (Why is that?)
• Entropy estimation in Turing's perspective
• Bayesian entropy estimation for countable discrete distributions

Cited In (10)

• Title not available (Why is that?)
• Estimating the entropy of binary time series: methodology, some theory and a simulation study
• Entropy estimates of small data sets
• A note on entropy optimization
• An asymptotic lower bound for the entropy of discrete populations with application to the estimation of entropy for approximately uniform populations
• Estimating entropy rate from censored symbolic time series: A test for time-irreversibility
• A proof of the estimation from below in Pesin's entropy formula
• Symbolic partition in chaotic maps
• Scaling behaviour of entropy estimates
• Entropy estimation in Turing's perspective

This page was built for publication: A note on entropy estimation

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5380327)