The permutation entropy rate equals the metric entropy rate for ergodic information sources and ergodic dynamical systems

DOI10.1016/J.PHYSD.2005.07.006zbMATH Open1149.37300arXivnlin/0503044OpenAlexW2006980549MaRDI QIDQ2509367FDOQ2509367

Authors: Matthew B. Kennel, Ljupco Kocarev, José M. Amigó 

Publication date: 19 October 2006

Published in: Physica D (Search for Journal in Brave)

Abstract: Permutation entropy quantifies the diversity of possible orderings of the values a random or deterministic system can take, as Shannon entropy quantifies the diversity of values. We show that the metric and permutation entropy rates--measures of new disorder per new observed value--are equal for ergodic finite-alphabet information sources (discrete-time stationary stochastic processes). With this result, we then prove that the same holds for deterministic dynamical systems defined by ergodic maps on $n$% -dimensional intervals. This result generalizes a previous one for piecewise monotone interval maps on the real line (Bandt, Keller and Pompe, "Entropy of interval maps via permutations", extit{Nonlinearity} extbf{15}, 1595-602, (2002)), at the expense of requiring ergodicity and using a definition of permutation entropy rate differing in the order of two limits. The case of non-ergodic finite-alphabet sources is also studied and an inequality developed. Finally, the equality of permutation and metric entropy rates is extended to ergodic non-discrete information sources when entropy is replaced by differential entropy in the usual way.

Full work available at URL: https://arxiv.org/abs/nlin/0503044

Recommendations

• The equality of Kolmogorov-Sinai entropy and metric permutation entropy generalized
• A generalized permutation entropy for noisy dynamics and random processes
• A limit property of relative entropy density for arbitrary discrete information sources
• Permutation entropy revisited
• Entropic measures, Markov information sources and complexity
• scientific article; zbMATH DE number 3863086
• Some properties of the relative entropy density of arbitrary information source
• Entropy and Relative Entropy From Information-Theoretic Principles
• scientific article; zbMATH DE number 3294098
• Shannon entropy: a rigorous notion at the crossroads between probability, information theory, dynamical systems and statistical physics

zbMATH Keywords

entropysymbolic dynamicsKolmogorov-Sinai entropypermutation entropy

Mathematics Subject Classification ID

Information theory (general) (94A15) Measures of information, entropy (94A17) Entropy and other invariants, isomorphism, classification in ergodic theory (37A35) Dynamical systems and their relations with probability theory and stochastic processes (37A50)

Cites Work

• Title not available (Why is that?)
• Permutations and topological entropy for interval maps
• Title not available (Why is that?)
• Entropy of interval maps via permutations
• Compression of individual sequences via variable-rate coding
• On the Complexity of Finite Sequences
• Title not available (Why is that?)
• Estimating Entropy Rates with Bayesian Confidence Intervals
• Estimating the Entropy Rate of Spike Trains via Lempel-Ziv Complexity
• Nonparametric entropy estimation for stationary processes and random fields, with applications to English text
• Title not available (Why is that?)

Cited In (31)

• Entropy determination based on the ordinal structure of a dynamical system
• Kolmogorov-Sinai entropy via separation properties of order-generated \(\sigma\)-algebras
• Permutation entropy and its main biomedical and econophysics applications: a review
• The equality of Kolmogorov-Sinai entropy and metric permutation entropy generalized
• Forbidden ordinal patterns in higher dimensional dynamics
• The ordinal structure of the signed shift transformations
• Application of joint permutations for predicting coupled time series
• Permutation complexity and coupling measures in hidden Markov models
• Symbolic transfer entropy rate is equal to transfer entropy rate for bivariate finite-alphabet stationary ergodic Markov processes
• Kolmogorov-Sinai entropy from the ordinal viewpoint
• Partially ordered permutation entropies
• Permutation complexity via duality between values and orderings
• Forbidden patterns and shift systems
• Describing high-dimensional dynamics with low-dimensional piecewise affine models: Applications to renewable energy
• Analysis of bivariate coupling by means of recurrence
• On the relation of KS entropy and permutation entropy
• Composition law of cardinal ordering permutations
• Permutations uniquely identify states and unknown external forces in non-autonomous dynamical systems
• Transcripts: An algebraic approach to coupled time series
• Approximating high-dimensional dynamics by barycentric coordinates with linear programming
• DETECTING DETERMINISM IN TIME SERIES WITH ORDINAL PATTERNS: A COMPARATIVE STUDY
• Topological permutation entropy
• Order Patterns in Time Series
• TIME SERIES FROM THE ORDINAL VIEWPOINT
• Discrete entropy
• Time series classification based on triadic time series motifs
• Measuring the jitter of ring oscillators by means of information theory quantifiers
• Causation entropy from symbolic representations of dynamical systems
• A review of symbolic dynamics and symbolic reconstruction of dynamical systems
• Ordinal methods: concepts, applications, new developments, and challenges -- in memory of Karsten Keller (1961--2022)
• Complexity of couplings in multivariate time series via ordinal persistent homology

This page was built for publication: The permutation entropy rate equals the metric entropy rate for ergodic information sources and ergodic dynamical systems

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2509367)