A compositional framework for Markov processes

DOI10.1063/1.4941578MaRDI QIDQ2798694zbMATH OpenOpenAlexWikidataFDO

Authors John Baez, Brendan Fong, Blake S. Pollard

Publication date 13 April 2016

Published in Journal of Mathematical Physics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1508.06448

black box functor circuits Lagrangian relations Markov processes principle of minimum dissipation symplectic vector spaces

Mathematics Subject Classification ID

Symplectic manifolds (general theory) (53D05) Continuous-time Markov processes on general state spaces (60J25) Technical applications of optics and electromagnetic theory (78A55) Irreversible thermodynamics, including Onsager-Machlup theory (82B35) Measures of information, entropy (94A17)

Abstract: We define the concept of an "open" Markov process, or more precisely, continuous-time Markov chain, which is one where probability can flow in or out of certain states called "inputs" and "outputs". One can build up a Markov process from smaller open pieces. This process is formalized by making open Markov processes into the morphisms of a dagger compact category. We show that the behavior of a detailed balanced open Markov process is determined by a principle of minimum dissipation, closely related to Prigogine's principle of minimum entropy production. Using this fact, we set up a functor mapping open detailed balanced Markov processes to open circuits made of linear resistors. We also describe how to "black box" an open Markov process, obtaining the linear relation between input and output data that holds in any steady state, including nonequilibrium steady states with a nonzero flow of probability through the system. We prove that black boxing gives a symmetric monoidal dagger functor sending open detailed balanced Markov processes to Lagrangian relations between symplectic vector spaces. This allows us to compute the steady state behavior of an open detailed balanced Markov process from the behaviors of smaller pieces from which it is built. We relate this black box functor to a previously constructed black box functor for circuits.

Review text

A continuous-time Markov chain is a way to determine the dynamics of a population which is spread across some finite set of states. Population can flow between the states. Under certain conditions the population of the states tend towards an equilibrium in which at any state the inflow of population is equal to its outflow. In an electrical circuit of linear registors, charge can flow along wires, In equilibrium, without any driving voltage from outside, the current along each wire is zero, and the potential at each node is equal.NEWLINEA continuous-time Markov chains is called \textit{detailed balanced} if in equilibrium, for any two vertices connected by an edge, the flow from one of the two vertices to the other equals the opposite flow. The authors call continuous-time Markov chains \textit{Markov processes}, and the principal objective in this paper is to formalize and exploit the well-known analogy between detailed balanced Markov processes and electrical circuits of linear registors ($\text{potential}\Leftrightarrow\text{population}$, $\text{current}\Leftrightarrow\text{flow}$, $\text{conductance}\Leftrightarrow\text{rate constant}$, $\text{power}\Leftrightarrow\text{dissipation}$) [\textit{F. P. Kelly}, Reversibility and stochastic networks. Chichester etc.: John Wiley \& Sons (1979; Zbl 0422.60001); Reversibility and stochastic networks. With a new preface. Reprint of the 1979 ed. Cambridge: Cambridge University Press (2011; Zbl 1260.60001); \textit{J. F. C. Kingman}, J. Appl. Probab. 6, 1--18 (1969; Zbl 0177.21807); \textit{C. St. J. A. Nash-Williams}, Proc. Camb. Philos. Soc. 55, 181--194 (1959; Zbl 0100.13602)].NEWLINENEWLINEThe authors have studied electrical circuits by introducing a framework for \textit{black boxing} a circuit and extracting the relations it determines between potential-current pairs at the input amd output terminals [\textit{J. C. Baez} and \textit{B. Fong}, Theory Appl. Categ. 33, 1158--1222 (2018; Zbl 1402.18005)]. The relation depicts the external behavior as can be observed by someone who is to perform measurements at the terminals. It is significant that black boxing is \textit{compositional} in the sense that if one builds a circuit from smaller pieces, the external behavior of the entire circuit can be determined from those of the pieces. This paper exploits this framework to detailed balanced Markov processes.NEWLINENEWLINEThe paper consists of 14 sections together with a tutorial appendix on decorated cospans [\textit{B. Fong}, Theory Appl. Categ. 30, 1096--1120 (2015; Zbl 1351.18003)]. \S II is an overview of main ideas. \S III is a review of Markov processes. \S IV defines open Markov processes and the open master equation. \S V introduces the concept of detailed balance for open Markov processes. \S VI recalls the principle of minimum power for open circuits of linear registors and expounds how to black box them. \S VII introduces the principle of minimum dissipation for open detailed balanced\ Markov processes and explicate how to black box them. \S VIII claims the analogy between electrical circuits and detailed balanced\ Markov processes formally. \S IX describes how to compose open Markov processes, while \S X does the same for detailed balanced\ Markov processes. \S XI describes the black box functor sending any open detailed balanced\ Markov process to the linear relation which describes its external behavior. \S XII makes the analogy between open detailed balanced\ Markov processes and open circuits a functor. It is shown in \S XIII that the linear relations in the image of these black box functors are Lagrangian relations between symplectic vector spaces, being shown also that the master equation is to be seen as a gradient flow equation. \S XIV summarizes the main findings in the paper.

Recommendations

Cites work

Cited in

(28)

This page was built for publication: A compositional framework for Markov processes

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2798694)