Computing expected transition events in reducible Markov chains

DOI10.1137/080726082MaRDI QIDQ3584145zbMATH OpenOpenAlexFDO

Authors Brian Ewald, Jeffrey Humpherys, Jeremy M. West

Publication date 19 August 2010

Published in SIAM Journal on Matrix Analysis and Applications (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/0806.1291

zbMATH Keywords

Markov chainsreducible matricesgeneralized inverses

Mathematics Subject Classification ID

Computational methods in Markov chains (60J22) Theory of matrix inversion and generalized inverses (15A09)

Abstract: We present a closed-form, computable expression for the expected number of times any transition event occurs during the transient phase of a reducible Markov chain. Examples of events include time to absorption, number of visits to a state, traversals of a particular transition, loops from a state to itself, and arrivals to a state from a particular subset of states. We give an analogous expression for time-average events, which describe the steady-state behavior of reducible chains as well as the long-term behavior of irreducible chains.

Recommendations

• Reduction of absorbing Markov chain
• Computing mean first passage times for a Markov chain
• The expected hitting times for finite Markov chains
• Publication:4945337
• On a new method of Markov chain reduction

This page was built for publication: Computing expected transition events in reducible Markov chains

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3584145)