Information anatomy of stochastic equilibria

DOI10.3390/E16094713zbMATH Open1338.60209arXiv1403.3864OpenAlexW2075428716MaRDI QIDQ296492FDOQ296492

Sarah Marzen, James P. Crutchfield

Publication date: 15 June 2016

Published in: Entropy (Search for Journal in Brave)

Abstract: A stochastic nonlinear dynamical system generates information, as measured by its entropy rate. Some---the ephemeral information---is dissipated and some---the bound information---is actively stored and so affects future behavior. We derive analytic expressions for the ephemeral and bound informations in the limit of small-time discretization for two classical systems that exhibit dynamical equilibria: first-order Langevin equations (i) where the drift is the gradient of a potential function and the diffusion matrix is invertible and (ii) with a linear drift term (Ornstein-Uhlenbeck) but a noninvertible diffusion matrix. In both cases, the bound information is sensitive only to the drift, while the ephemeral information is sensitive only to the diffusion matrix and not to the drift. Notably, this information anatomy changes discontinuously as any of the diffusion coefficients vanishes, indicating that it is very sensitive to the noise structure. We then calculate the information anatomy of the stochastic cusp catastrophe and of particles diffusing in a heat bath in the overdamped limit, both examples of stochastic gradient descent on a potential landscape. Finally, we use our methods to calculate and compare approximations for the so-called time-local predictive information for adaptive agents.

Full work available at URL: https://arxiv.org/abs/1403.3864

Mathematics Subject Classification ID

Applications of Brownian motions and diffusion theory (population genetics, absorption problems, etc.) (60J70) Generation, random and stochastic difference and differential equations (37H10)

Cites Work

• Title not available (Why is that?)
• The Fokker-Planck equation. Methods of solutions and applications.
• Regularities unseen, randomness observed: Levels of entropy convergence
• Title not available (Why is that?)
• Prediction, retrodiction, and the amount of information stored in the present
• Predictive information in a nonequilibrium critical model
• Stability and diversity in collective adaptation
• Information theory and network coding
• Title not available (Why is that?)
• Synchronization and control in intrinsic and designed computation: An information-theoretic analysis of competing models of stochastic computation
• Information modification and particle collisions in distributed computation
• Partial information decomposition as a spatiotemporal filter
• Anatomy of a bit: Information in a time series observation
• ∊-Machine spectral reconstruction theory: a direct method for inferring planar disorder and structure from X-ray diffraction studies
• Computational mechanics: pattern and prediction, structure and simplicity.
• Properties of the statistical complexity functional and partially deterministic HMMs
• Thermodynamic formalism for systems with Markov dynamics

Cited In (5)

• Assessing pandemic uncertainty on conditions of vaccination and self-isolation
• A Pandemic Three-Sided Coin
• Predictive rate-distortion for infinite-order Markov processes
• Informational and causal architecture of continuous-time renewal processes
• Uncertainty in epidemic models based on a three-sided coin

This page was built for publication: Information anatomy of stochastic equilibria

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q296492)