Algorithms for determining the state-time probabilities and the limit matrix in Markov chains

MaRDI QIDQ3087469zbMATH OpenFDO

Authors Stefan Pickl, Dmitrii Lozovanu

Publication date 16 August 2011

Full work available at URL http://www.math.md/publications/basm/issues/y2011-n1/10841/

zbMATH Keywords

discrete Markov processdynamic programminglimit state matrixpolynomial time algorithmprobability of state transition

Mathematics Subject Classification ID

Dynamic programming in optimal control and differential games (49L20) Markov chains (discrete-time Markov processes on discrete state spaces) (60J10) Optimal stochastic control (93E20)

Recommendations

• A dynamic programming approach for finite Markov processes and algorithms for the calculation of the limit matrix in Markov chains
• Algorithms for Markov stochastic models
• An approach for determining the matrix of limiting state probabilities in discrete Markov processes.
• On an algorithm for calculating stationary probabilities of the Markov chains
• scientific article; zbMATH DE number 7329538

Cited in

(18)

• Algorithms for an irreducible and lumpable strong stochastic bound
• An effective Markov based approach for calculating the limit matrix in the analytic network process
• Computer aided solving the high-order transition probability matrix of the finite Markov chain
• On an algorithm for calculating stationary probabilities of the Markov chains
• Algorithms for determining the transient and differential matrices in finite Markov processes
• A dynamic programming approach for finite Markov processes and algorithms for the calculation of the limit matrix in Markov chains
• Fully polynomial-time computation of maximum likelihood trajectories in Markov chains
• Computational discrete time Markov chain with correlated transition probabilities
• An approach for determining the matrix of limiting state probabilities in discrete Markov processes.
• scientific article; zbMATH DE number 19236 (Why is no real title available?)
• Generating Markov-Chain Transitions Quickly: II
• scientific article; zbMATH DE number 549928 (Why is no real title available?)
• scientific article; zbMATH DE number 1786171 (Why is no real title available?)
• Algorithm of state stationary probability computing for continuous-time finite Markov chain modulated by semi-Markov process
• A probabilistic algorithm for finding the rate matrix of a block-GI/M/1 Markov chain
• Calculation of higher transitions in a Markov process
• scientific article; zbMATH DE number 7329538 (Why is no real title available?)
• Algorithms for Markov stochastic models

This page was built for publication: Algorithms for determining the state-time probabilities and the limit matrix in Markov chains

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3087469)