Technical Note—A Markov Chain Partitioning Algorithm for Computing Steady State Probabilities

DOI10.1287/OPRE.33.1.228zbMATH Open0569.90092OpenAlexW1966781403MaRDI QIDQ3686459FDOQ3686459

Authors: Theodore J. Sheskin 

Publication date: 1985

Published in: Operations Research (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1287/opre.33.1.228

Recommendations

• A partitioning algorithm for computing limiting distributions of ill-conditioned Markov chains
• Algorithms for finding steady state probabilities for some special classes of finite state Markov chains
• On an algorithm for calculating stationary probabilities of the Markov chains
• A Method to Calculate Steady-State Distributions of Large Markov Chains by Aggregating States
• Probabilistic approach to computational algorithms for finding stationary distributions of Markov chains
• Computation of the steady-state probability of Markov chain evolving on a mixed state space
• A Stable Algorithm to Calculate Steady-State Probability & Frequency of a Markov System
• An aggregation/disaggregation algorithm for computing the stationary distribution of a large markov chain

zbMATH Keywords

Markov processpartitioning algorithmfinite, irreducible Markov chainmatrix reduction routinerecursively computing the steady state probabilities

Mathematics Subject Classification ID

Markov renewal processes, semi-Markov processes (60K15) Continuous-time Markov processes on discrete state spaces (60J27) Markov and semi-Markov decision processes (90C40)

Cited In (31)

• Accurate calculations of stationary distributions and mean first passage times in Markov renewal processes and Markov chains
• Computational discrete time Markov chain with correlated transition probabilities
• A partitioning algorithm for solving systems of linear equations
• A Consistent Markov Partition Process Generated from the Paintbox Process
• Steady-State Approximation for a Vector Valued Markov Chain
• The Markov chain tree theorem in commutative semirings and the state reduction algorithm in commutative semifields
• State reduction in a Markov decision process
• Comparacion numerica de algoritmos para calcular distribuciones estacionarias de cadenas de Markov finitas
• Optimal stopping of seasonal observations and projection of a Markov chain
• A multi-cluster time aggregation approach for Markov chains
• Computing absorption probabilities for a Markov chain
• Computing mean first passage times for a Markov chain
• Parallel computing for Markov chains with islands and ports
• Asymptotic expansions for stationary distributions of perturbed semi-Markov processes
• A partitioning algorithm for computing limiting distributions of ill-conditioned Markov chains
• Computing the fundamental matrix for a nonirreducible Markov chain
• The computation of key properties of Markov chains via perturbations
• Aggregation of Markov flows I: theory
• Computation of the steady-state probability of Markov chain evolving on a mixed state space
• Computing hitting probabilities of Markov chains: structural results with regard to the solution space of the corresponding system of equations
• A Method to Calculate Steady-State Distributions of Large Markov Chains by Aggregating States
• A kind of dual form for coupling from the past algorithm, to sample from Markov chain steady-state probability
• Cartesian product partitioning of multi-dimensional reachable state spaces
• The state reduction and related algorithms and their applications to the study of Markov chains, graph theory, and the optimal stopping problem
• The computation of the mean first passage times for Markov chains
• Matrix inversion by augmentation and reduction
• Perfect simulation of steady-state Markov chain on mixed state space
• Algorithms for finding steady state probabilities for some special classes of finite state Markov chains
• A Stable Algorithm to Calculate Steady-State Probability & Frequency of a Markov System
• Reduced System Algorithms for Markov Chains
• Susceptible-exposed-infectious model using Markov chains

This page was built for publication: Technical Note—A Markov Chain Partitioning Algorithm for Computing Steady State Probabilities

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3686459)