Information symmetries in irreversible processes

DOI10.1063/1.3637490zbMATH Open1317.60088arXiv1107.2168OpenAlexW2078344066WikidataQ46219403 ScholiaQ46219403MaRDI QIDQ5264345FDOQ5264345

Jörg Reichardt, John R. Mahoney, James P. Crutchfield, Ryan G. James, Christopher J. Ellison

Publication date: 27 July 2015

Published in: Chaos: An Interdisciplinary Journal of Nonlinear Science (Search for Journal in Brave)

Abstract: We study dynamical reversibility in stationary stochastic processes from an information theoretic perspective. Extending earlier work on the reversibility of Markov chains, we focus on finitary processes with arbitrarily long conditional correlations. In particular, we examine stationary processes represented or generated by edge-emitting, finite-state hidden Markov models. Surprisingly, we find pervasive temporal asymmetries in the statistics of such stationary processes with the consequence that the computational resources necessary to generate a process in the forward and reverse temporal directions are generally not the same. In fact, an exhaustive survey indicates that most stationary processes are irreversible. We study the ensuing relations between model topology in different representations, the process's statistical properties, and its reversibility in detail. A process's temporal asymmetry is efficiently captured using two canonical unifilar representations of the generating model, the forward-time and reverse-time epsilon-machines. We analyze example irreversible processes whose epsilon-machine presentations change size under time reversal, including one which has a finite number of recurrent causal states in one direction, but an infinite number in the opposite. From the forward-time and reverse-time epsilon-machines, we are able to construct a symmetrized, but nonunifilar, generator of a process---the bidirectional machine. Using the bidirectional machine, we show how to directly calculate a process's fundamental information properties, many of which are otherwise only poorly approximated via process samples. The tools we introduce and the insights we offer provide a better understanding of the many facets of reversibility and irreversibility in stochastic processes.

Full work available at URL: https://arxiv.org/abs/1107.2168

Recommendations

• Time-reversed dynamical entropy and irreversibility in Markovian random processes
• scientific article; zbMATH DE number 3930061
• scientific article; zbMATH DE number 3954011
• Publication:4731930
• About the reversibility and irreversibility of stochastic systems

Mathematics Subject Classification ID

Numerical analysis or methods applied to Markov chains (65C40) Markov chains (discrete-time Markov processes on discrete state spaces) (60J10)

Cites Work

• Elements of Information Theory
• Hidden Markov processes
• Title not available (Why is that?)
• Subshifts of finite type and sofic systems
• Title not available (Why is that?)
• An Introduction to Symbolic Dynamics and Coding
• Statistical Inference for Probabilistic Functions of Finite State Markov Chains
• Identifiability of hidden Markov information sources and their minimum degrees of freedom
• On the Identifiability Problem for Functions of Finite Markov Chains
• Toward a quantitative theory of self-generated complexity
• Predictability, complexity, and learning
• Regularities unseen, randomness observed: Levels of entropy convergence
• Inference of Reversible Languages
• Nonequilibrium measurements of free energy differences for microscopically reversible Markovian systems
• Prediction, retrodiction, and the amount of information stored in the present
• Synchronization and control in intrinsic and designed computation: An information-theoretic analysis of competing models of stochastic computation
• Anatomy of a bit: Information in a time series observation
• Computational mechanics: pattern and prediction, structure and simplicity.
• The calculi of emergence: Computation, dynamics and induction
• Information accessibility and cryptic processes
• On locally reversible languages
• Sofic systems
• ON THE GENERATIVE NATURE OF PREDICTION
• A new outlook on Shannon's information measures
• The structure of the I-measure of a Markov chain
• QUANTIFYING EMERGENCE IN TERMS OF PERSISTENT MUTUAL INFORMATION
• How hidden are hidden processes? A primer on crypticity and entropy convergence

Cited In (7)

• Symmetry in information flow
• The fundamental thermodynamic bounds on finite models
• Predictive rate-distortion for infinite-order Markov processes
• On the information balance in a simple chemical process
• Title not available (Why is that?)
• Information criteria for quantifying loss of reversibility in parallelized KMC
• How hidden are hidden processes? A primer on crypticity and entropy convergence

This page was built for publication: Information symmetries in irreversible processes

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5264345)