The permutation entropy rate equals the metric entropy rate for ergodic information sources and ergodic dynamical systems

DOI10.1016/J.PHYSD.2005.07.006MaRDI QIDQ2509367zbMATH OpenOpenAlexFDO

Authors Matthew B. Kennel, Ljupco Kocarev, José M. Amigó

Publication date 19 October 2006

Published in Physica D (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/nlin/0503044

zbMATH Keywords

entropysymbolic dynamicsKolmogorov-Sinai entropypermutation entropy

Mathematics Subject Classification ID

Information theory (general) (94A15) Measures of information, entropy (94A17) Entropy and other invariants, isomorphism, classification in ergodic theory (37A35) Dynamical systems and their relations with probability theory and stochastic processes (37A50)

Abstract: Permutation entropy quantifies the diversity of possible orderings of the values a random or deterministic system can take, as Shannon entropy quantifies the diversity of values. We show that the metric and permutation entropy rates--measures of new disorder per new observed value--are equal for ergodic finite-alphabet information sources (discrete-time stationary stochastic processes). With this result, we then prove that the same holds for deterministic dynamical systems defined by ergodic maps on $n$% -dimensional intervals. This result generalizes a previous one for piecewise monotone interval maps on the real line (Bandt, Keller and Pompe, "Entropy of interval maps via permutations", extit{Nonlinearity} extbf{15}, 1595-602, (2002)), at the expense of requiring ergodicity and using a definition of permutation entropy rate differing in the order of two limits. The case of non-ergodic finite-alphabet sources is also studied and an inequality developed. Finally, the equality of permutation and metric entropy rates is extended to ergodic non-discrete information sources when entropy is replaced by differential entropy in the usual way.

Recommendations

• The equality of Kolmogorov-Sinai entropy and metric permutation entropy generalized
• A generalized permutation entropy for noisy dynamics and random processes
• A limit property of relative entropy density for arbitrary discrete information sources
• Permutation entropy revisited
• Entropic measures, Markov information sources and complexity
• scientific article; zbMATH DE number 3863086
• Some properties of the relative entropy density of arbitrary information source
• Entropy and Relative Entropy From Information-Theoretic Principles
• scientific article; zbMATH DE number 3294098
• Shannon entropy: a rigorous notion at the crossroads between probability, information theory, dynamical systems and statistical physics

Cites work

• scientific article; zbMATH DE number 48436 (Why is no real title available?)
• scientific article; zbMATH DE number 107482 (Why is no real title available?)
• scientific article; zbMATH DE number 1216169 (Why is no real title available?)
• scientific article; zbMATH DE number 1542660 (Why is no real title available?)
• Compression of individual sequences via variable-rate coding
• Entropy of interval maps via permutations
• Estimating Entropy Rates with Bayesian Confidence Intervals
• Estimating the Entropy Rate of Spike Trains via Lempel-Ziv Complexity
• Nonparametric entropy estimation for stationary processes and random fields, with applications to English text
• On the Complexity of Finite Sequences
• Permutations and topological entropy for interval maps

Cited in

(31)

• Entropy determination based on the ordinal structure of a dynamical system
• Kolmogorov-Sinai entropy via separation properties of order-generated \(\sigma\)-algebras
• Permutation entropy and its main biomedical and econophysics applications: a review
• The equality of Kolmogorov-Sinai entropy and metric permutation entropy generalized
• Forbidden ordinal patterns in higher dimensional dynamics
• Permutation complexity and coupling measures in hidden Markov models
• Application of joint permutations for predicting coupled time series
• The ordinal structure of the signed shift transformations
• Kolmogorov-Sinai entropy from the ordinal viewpoint
• Symbolic transfer entropy rate is equal to transfer entropy rate for bivariate finite-alphabet stationary ergodic Markov processes
• Permutation complexity via duality between values and orderings
• Partially ordered permutation entropies
• Forbidden patterns and shift systems
• Describing high-dimensional dynamics with low-dimensional piecewise affine models: Applications to renewable energy
• Analysis of bivariate coupling by means of recurrence
• Composition law of cardinal ordering permutations
• On the relation of KS entropy and permutation entropy
• Permutations uniquely identify states and unknown external forces in non-autonomous dynamical systems
• Transcripts: An algebraic approach to coupled time series
• Approximating high-dimensional dynamics by barycentric coordinates with linear programming
• DETECTING DETERMINISM IN TIME SERIES WITH ORDINAL PATTERNS: A COMPARATIVE STUDY
• Topological permutation entropy
• Order Patterns in Time Series
• TIME SERIES FROM THE ORDINAL VIEWPOINT
• Discrete entropy
• Measuring the jitter of ring oscillators by means of information theory quantifiers
• Time series classification based on triadic time series motifs
• Causation entropy from symbolic representations of dynamical systems
• A review of symbolic dynamics and symbolic reconstruction of dynamical systems
• Ordinal methods: concepts, applications, new developments, and challenges -- in memory of Karsten Keller (1961--2022)
• Complexity of couplings in multivariate time series via ordinal persistent homology

This page was built for publication: The permutation entropy rate equals the metric entropy rate for ergodic information sources and ergodic dynamical systems

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2509367)