How hidden are hidden processes? A primer on crypticity and entropy convergence

DOI10.1063/1.3637502MaRDI QIDQ5264348zbMATH OpenOpenAlexWikidataFDO

Authors John R. Mahoney, Christopher J. Ellison, Ryan G. James, James P. Crutchfield

Publication date 27 July 2015

Published in Chaos: An Interdisciplinary Journal of Nonlinear Science (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1108.1510

Mathematics Subject Classification ID

Measures of information, entropy (94A17) Graph minors (05C83)

Abstract: We investigate a stationary process's crypticity---a measure of the difference between its hidden state information and its observed information---using the causal states of computational mechanics. Here, we motivate crypticity and cryptic order as physically meaningful quantities that monitor how hidden a hidden process is. This is done by recasting previous results on the convergence of block entropy and block-state entropy in a geometric setting, one that is more intuitive and that leads to a number of new results. For example, we connect crypticity to how an observer synchronizes to a process. We show that the block-causal-state entropy is a convex function of block length. We give a complete analysis of spin chains. We present a classification scheme that surveys stationary processes in terms of their possible cryptic and Markov orders. We illustrate related entropy convergence behaviors using a new form of foliated information diagram. Finally, along the way, we provide a variety of interpretations of crypticity and cryptic order to establish their naturalness and pervasiveness. Hopefully, these will inspire new applications in spatially extended and network dynamical systems.

Recommendations

• Information accessibility and cryptic processes
• On hidden Markov processes with infinite excess entropy
• Entropy of hidden Markov processes via cycle expansion
• Regularities unseen, randomness observed: Levels of entropy convergence
• Shannon entropy rate of hidden Markov processes

Cites work

• A new outlook on Shannon's information measures
• Anatomy of a bit: information in a time series observation
• Computational mechanics: pattern and prediction, structure and simplicity.
• Information accessibility and cryptic processes
• Information symmetries in irreversible processes
• Prediction, retrodiction, and the amount of information stored in the present
• Regularities unseen, randomness observed: Levels of entropy convergence
• SYNCHRONIZING TO THE ENVIRONMENT: INFORMATION-THEORETIC CONSTRAINTS ON AGENT LEARNING
• Synchronization and control in intrinsic and designed computation: An information-theoretic analysis of competing models of stochastic computation

Cited in

(5)

• Information accessibility and cryptic processes
• Spectral simplicity of apparent complexity. I: The nondiagonalizable metadynamics of prediction
• Information symmetries in irreversible processes
• The fundamental thermodynamic bounds on finite models
• Synchronization and control in intrinsic and designed computation: An information-theoretic analysis of competing models of stochastic computation

This page was built for publication: How hidden are hidden processes? A primer on crypticity and entropy convergence

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5264348)