Numerical Solution of Linear Equations Arising in Markov Chain Models

DOI10.1287/IJOC.1.1.52MaRDI QIDQ4018153zbMATH OpenOpenAlexFDO

Authors Daniel P. Heyman, Alyson Reeves

Publication date 16 January 1993

Published in ORSA Journal on Computing (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1287/ijoc.1.1.52

zbMATH Keywords

comparison of methodsfirst-passage timesMarkov chainsnon-iterative methodsstate-reductionsubstochastic matrix

Mathematics Subject Classification ID

Markov chains (discrete-time Markov processes on discrete state spaces) (60J10) Probabilistic methods, stochastic differential equations (65C99) Direct numerical methods for linear systems and matrix inversion (65F05)

Recommendations

• scientific article; zbMATH DE number 729460
• scientific article; zbMATH DE number 4009652
• On the numerical solution of a nonlinear matrix equation in Markov chains
• Error Bounds for the Computation of Null Vectors with Applications to Markov Chains
• Publication:3484088

Cited in

(26)

• Using state reduction for computing steady state vectors in Markov chains of \(M/G/1\) type
• Optimizing steady state Markov chains by state reduction
• The state reduction and related algorithms and their applications to the study of Markov chains, graph theory, and the optimal stopping problem
• Numerical evaluation of resolvents and Laplace transforms of Markov processes using linear programming
• The computation of the mean first passage times for Markov chains
• A numerical algorithm on the computation of the stationary distribution of a discrete time homogenous finite Markov chain
• Development of computational algorithm for multiserver queue with renewal input and synchronous vacation
• Stationary distributions and mean first passage times of perturbed Markov chains
• Accurate calculations of stationary distributions and mean first passage times in Markov renewal processes and Markov chains
• scientific article; zbMATH DE number 5929168 (Why is no real title available?)
• scientific article; zbMATH DE number 6118049 (Why is no real title available?)
• State reduction in a Markov decision process
• The computation of key properties of Markov chains via perturbations
• Influence of Preconditioning and Blocking on Accuracy in Solving Markovian Models
• scientific article; zbMATH DE number 4155539 (Why is no real title available?)
• scientific article; zbMATH DE number 5177456 (Why is no real title available?)
• Numerical computation of mean passage times and absorption probabilities in Markov and Semi-Markov models
• Numerical solution of sparse singular systems of equations arising from ergodic markov chains
• scientific article; zbMATH DE number 19249 (Why is no real title available?)
• Numerical Methods in Markov Chain Modeling
• Computing the Decay of a Simple Reversible Sub-Markov Semigroup
• Numerical Solution of a Matrix Integral Equation Arising in Markov-Modulated Lévy Processes
• Robust power series algorithm for epistemic uncertainty propagation in Markov chain models
• An algorithmic approach for a special class of Markov chains
• Solution methods for discrete-state Markovian initial value problems
• Variances of first passage times in a Markov chain with applications to mixing times

This page was built for publication: Numerical Solution of Linear Equations Arising in Markov Chain Models

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4018153)