How hidden are hidden processes? A primer on crypticity and entropy convergence

DOI10.1063/1.3637502zbMATH Open1317.94040arXiv1108.1510OpenAlexW3103122548WikidataQ46666503 ScholiaQ46666503MaRDI QIDQ5264348FDOQ5264348

Authors: John R. Mahoney, Christopher J. Ellison, Ryan G. James, James P. Crutchfield 

Publication date: 27 July 2015

Published in: Chaos: An Interdisciplinary Journal of Nonlinear Science (Search for Journal in Brave)

Abstract: We investigate a stationary process's crypticity---a measure of the difference between its hidden state information and its observed information---using the causal states of computational mechanics. Here, we motivate crypticity and cryptic order as physically meaningful quantities that monitor how hidden a hidden process is. This is done by recasting previous results on the convergence of block entropy and block-state entropy in a geometric setting, one that is more intuitive and that leads to a number of new results. For example, we connect crypticity to how an observer synchronizes to a process. We show that the block-causal-state entropy is a convex function of block length. We give a complete analysis of spin chains. We present a classification scheme that surveys stationary processes in terms of their possible cryptic and Markov orders. We illustrate related entropy convergence behaviors using a new form of foliated information diagram. Finally, along the way, we provide a variety of interpretations of crypticity and cryptic order to establish their naturalness and pervasiveness. Hopefully, these will inspire new applications in spatially extended and network dynamical systems.

Full work available at URL: https://arxiv.org/abs/1108.1510

Recommendations

• Information accessibility and cryptic processes
• On hidden Markov processes with infinite excess entropy
• Entropy of hidden Markov processes via cycle expansion
• Regularities unseen, randomness observed: Levels of entropy convergence
• Shannon entropy rate of hidden Markov processes

Mathematics Subject Classification ID

Measures of information, entropy (94A17) Graph minors (05C83)

Cites Work

• Regularities unseen, randomness observed: Levels of entropy convergence
• Prediction, retrodiction, and the amount of information stored in the present
• SYNCHRONIZING TO THE ENVIRONMENT: INFORMATION-THEORETIC CONSTRAINTS ON AGENT LEARNING
• Synchronization and control in intrinsic and designed computation: An information-theoretic analysis of competing models of stochastic computation
• Anatomy of a bit: information in a time series observation
• Computational mechanics: pattern and prediction, structure and simplicity.
• Information symmetries in irreversible processes
• Information accessibility and cryptic processes
• A new outlook on Shannon's information measures

Cited In (5)

• The fundamental thermodynamic bounds on finite models
• Synchronization and control in intrinsic and designed computation: An information-theoretic analysis of competing models of stochastic computation
• Information symmetries in irreversible processes
• Information accessibility and cryptic processes
• Spectral simplicity of apparent complexity. I: The nondiagonalizable metadynamics of prediction

This page was built for publication: How hidden are hidden processes? A primer on crypticity and entropy convergence

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5264348)