Information symmetries in irreversible processes

DOI10.1063/1.3637490MaRDI QIDQ5264345zbMATHOpenAlexWikidataFDO

Authors Christopher J. Ellison, John R. Mahoney, Ryan G. James, James P. Crutchfield, Jörg Reichardt

Publication date 27 July 2015

Published in Chaos: An Interdisciplinary Journal of Nonlinear Science (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1107.2168

Mathematics Subject Classification ID

Numerical analysis or methods applied to Markov chains (65C40) Markov chains (discrete-time Markov processes on discrete state spaces) (60J10)

Abstract: We study dynamical reversibility in stationary stochastic processes from an information theoretic perspective. Extending earlier work on the reversibility of Markov chains, we focus on finitary processes with arbitrarily long conditional correlations. In particular, we examine stationary processes represented or generated by edge-emitting, finite-state hidden Markov models. Surprisingly, we find pervasive temporal asymmetries in the statistics of such stationary processes with the consequence that the computational resources necessary to generate a process in the forward and reverse temporal directions are generally not the same. In fact, an exhaustive survey indicates that most stationary processes are irreversible. We study the ensuing relations between model topology in different representations, the process's statistical properties, and its reversibility in detail. A process's temporal asymmetry is efficiently captured using two canonical unifilar representations of the generating model, the forward-time and reverse-time epsilon-machines. We analyze example irreversible processes whose epsilon-machine presentations change size under time reversal, including one which has a finite number of recurrent causal states in one direction, but an infinite number in the opposite. From the forward-time and reverse-time epsilon-machines, we are able to construct a symmetrized, but nonunifilar, generator of a process---the bidirectional machine. Using the bidirectional machine, we show how to directly calculate a process's fundamental information properties, many of which are otherwise only poorly approximated via process samples. The tools we introduce and the insights we offer provide a better understanding of the many facets of reversibility and irreversibility in stochastic processes.

Recommendations

• Time-reversed dynamical entropy and irreversibility in Markovian random processes
• scientific article; zbMATH DE number 3930061
• scientific article; zbMATH DE number 3954011
• Publication:4731930
• About the reversibility and irreversibility of stochastic systems

Cites work

• scientific article; zbMATH DE number 3664335 (Why is no real title available?)
• scientific article; zbMATH DE number 846906 (Why is no real title available?)
• A new outlook on Shannon's information measures
• An Introduction to Symbolic Dynamics and Coding
• Anatomy of a bit: information in a time series observation
• Computational mechanics: pattern and prediction, structure and simplicity.
• Elements of Information Theory
• Hidden Markov processes
• How hidden are hidden processes? A primer on crypticity and entropy convergence
• Identifiability of hidden Markov information sources and their minimum degrees of freedom
• Inference of Reversible Languages
• Information accessibility and cryptic processes
• Nonequilibrium measurements of free energy differences for microscopically reversible Markovian systems
• ON THE GENERATIVE NATURE OF PREDICTION
• On locally reversible languages
• On the Identifiability Problem for Functions of Finite Markov Chains
• Predictability, complexity, and learning
• Prediction, retrodiction, and the amount of information stored in the present
• Quantifying emergence in terms of persistent mutual information
• Regularities unseen, randomness observed: Levels of entropy convergence
• Sofic systems
• Statistical Inference for Probabilistic Functions of Finite State Markov Chains
• Subshifts of finite type and sofic systems
• Synchronization and control in intrinsic and designed computation: An information-theoretic analysis of competing models of stochastic computation
• The calculi of emergence: Computation, dynamics and induction
• The structure of the I-measure of a Markov chain
• Toward a quantitative theory of self-generated complexity

Cited in

(7)

• On the information balance in a simple chemical process
• Symmetry in information flow
• Predictive rate-distortion for infinite-order Markov processes
• Information criteria for quantifying loss of reversibility in parallelized KMC
• The fundamental thermodynamic bounds on finite models
• scientific article; zbMATH DE number 3871178 (Why is no real title available?)
• How hidden are hidden processes? A primer on crypticity and entropy convergence

This page was built for publication: Information symmetries in irreversible processes

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5264345)