Entropy estimation of symbol sequences
From MaRDI portal
Publication:4526375
DOI10.1063/1.166191zbMATH Open1055.94508arXivcond-mat/0203436OpenAlexW2018891628WikidataQ34202017 ScholiaQ34202017MaRDI QIDQ4526375FDOQ4526375
Authors: Thomas Schürmann, P. Grassberger 
Publication date: 16 January 2001
Published in: Chaos: An Interdisciplinary Journal of Nonlinear Science (Search for Journal in Brave)
Abstract: We discuss algorithms for estimating the Shannon entropy h of finite symbol sequences with long range correlations. In particular, we consider algorithms which estimate h from the code lengths produced by some compression algorithm. Our interest is in describing their convergence with sequence length, assuming no limits for the space and time complexities of the compression algorithms. A scaling law is proposed for extrapolation from finite sample lengths. This is applied to sequences of dynamical systems in non-trivial chaotic regimes, a 1-D cellular automaton, and to written English texts.
Full work available at URL: https://arxiv.org/abs/cond-mat/0203436
Recommendations
Protein sequences, DNA sequences (92D20) Measures of information, entropy (94A17) Dynamical aspects of cellular automata (37B15) Applications of dynamical systems (37N99)
Cites Work
- On Information and Sufficiency
- Stochastic complexity and modeling
- Compression of individual sequences via variable-rate coding
- Prediction and Entropy of Printed English
- Quantitative universality for a class of nonlinear transformations
- On the Complexity of Finite Sequences
- On the Length of Programs for Computing Finite Binary Sequences
- Logical basis for information theory and probability theory
- A universal data compression system
- The performance of universal encoding
- Universal schemes for prediction, gambling and portfolio selection
- The context-tree weighting method: basic properties
- Random sequence generation by cellular automata
- On the symbolic dynamics of the Henon map
- A universal finite memory source
- Universal modeling and coding
- Entropy and prefixes
- The redundancy of texts in three languages
- A convergent gambling estimate of the entropy of English
- LONG RANGE CORRELATION IN HUMAN WRITINGS
- Enlarged scaling ranges for the KS‐entropy and the information dimension
- A note on the Ziv - Lempel model for compressing individual sequences (Corresp.)
Cited In (42)
- A note on entropy estimation
- Symbolic dynamics of jejunal motility in the irritable bowel
- Large alphabets and incompressibility
- Estimating the entropy of binary time series: methodology, some theory and a simulation study
- SYNCHRONIZING TO THE ENVIRONMENT: INFORMATION-THEORETIC CONSTRAINTS ON AGENT LEARNING
- Modified correlation entropy estimation for a noisy chaotic time series
- Exit-times and \(\varepsilon\)-entropy for dynamical systems, stochastic processes, and turbulence
- Variance of entropy for testing time-varying regimes with an application to meme stocks
- Coincidences and estimation of entropies of random variables with large cardinalities
- On the non-randomness of maximum Lempel Ziv complexity sequences of finite size
- Regularities unseen, randomness observed: Levels of entropy convergence
- Non-sequential recursive pair substitution: some rigorous results
- Complexity through nonextensivity
- Towards the quantification of the semantic information encoded in written language
- Title not available (Why is that?)
- Contrasting stochasticity with chaos in a permutation Lempel-Ziv complexity -- Shannon entropy plane
- Word frequency and entropy of symbolic sequences: A dynamical perspective
- Finite sample effects in sequence analysis
- Estimating Information Rates with Confidence Intervals in Neural Spike Trains
- Entropy of high-order Markov chains beyond the pair correlations
- Title not available (Why is that?)
- BIAS REDUCTION OF THE NEAREST NEIGHBOR ENTROPY ESTIMATOR
- Convergence properties of functional estimates for discrete distributions
- Entropy analysis of word-length series of natural language texts: effects of text language and genre
- Possible sets of autocorrelations and the Simplex algorithm
- Non-sequential recursive pair substitutions and numerical entropy estimates in symbolic dynamical systems
- Estimating Entropy Rates with Bayesian Confidence Intervals
- Estimating entropy rate from censored symbolic time series: A test for time-irreversibility
- Guessing probability distributions from small samples
- Entropy of underdetermined sequences under constraints to specifications
- GeoEntropy: a measure of complexity and similarity
- Symbolic partition in chaotic maps
- Price predictability at ultra-high frequency: entropy-based randomness test
- Transfer entropy on symbolic recurrences
- Phenomenology of coupled nonlinear oscillators
- Forecasting the unemployment rate over districts with the use of distinct methods
- Compactness of symbolic sequences from chaotic systems
- Title not available (Why is that?)
- Maximally predictive states: from partial observations to long timescales
- Computational capabilities at the edge of chaos for one dimensional systems undergoing continuous transitions
- Optimal instruments and models for noisy chaos
- Artificial sequences and complexity measures
This page was built for publication: Entropy estimation of symbol sequences
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4526375)
