Computing expected transition events in reducible Markov chains

DOI10.1137/080726082zbMATH Open1196.60133arXiv0806.1291OpenAlexW2154159810MaRDI QIDQ3584145FDOQ3584145

Authors: Brian Ewald, Jeffrey Humpherys, Jeremy M. West 

Publication date: 19 August 2010

Published in: SIAM Journal on Matrix Analysis and Applications (Search for Journal in Brave)

Abstract: We present a closed-form, computable expression for the expected number of times any transition event occurs during the transient phase of a reducible Markov chain. Examples of events include time to absorption, number of visits to a state, traversals of a particular transition, loops from a state to itself, and arrivals to a state from a particular subset of states. We give an analogous expression for time-average events, which describe the steady-state behavior of reducible chains as well as the long-term behavior of irreducible chains.

Full work available at URL: https://arxiv.org/abs/0806.1291

Recommendations

• Reduction of absorbing Markov chain
• Computing mean first passage times for a Markov chain
• The expected hitting times for finite Markov chains
• Publication:4945337
• On a new method of Markov chain reduction

zbMATH Keywords

Markov chainsreducible matricesgeneralized inverses

Mathematics Subject Classification ID

Computational methods in Markov chains (60J22) Theory of matrix inversion and generalized inverses (15A09)

This page was built for publication: Computing expected transition events in reducible Markov chains

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3584145)