A dynamic programming approach for finite Markov processes and algorithms for the calculation of the limit matrix in Markov chains

DOI10.1080/02331934.2011.627333MaRDI QIDQ3111140zbMATH OpenOpenAlexFDO

Authors Stefan Pickl, Dmitrii Lozovanu

Publication date 18 January 2012

Published in Optimization (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1080/02331934.2011.627333

zbMATH Keywords

polynomial time algorithmdiscrete Markov processlimit state matrixprobability of state transitiondynamic prgramming

Mathematics Subject Classification ID

Computational methods in Markov chains (60J22) Numerical analysis or methods applied to Markov chains (65C40) Markov chains (discrete-time Markov processes on discrete state spaces) (60J10) Dynamic programming (90C39)

Recommendations

• Algorithms for determining the state-time probabilities and the limit matrix in Markov chains
• An approach for determining the matrix of limiting state probabilities in discrete Markov processes.
• scientific article; zbMATH DE number 7329538
• Algorithms for determining the transient and differential matrices in finite Markov processes
• Graphes et algorithme de calcul de probabilités stationnaires d'un processus markovien discret. (Graphs and algorithm for the computation of stationary probabilities of a discrete Markov process)

Cites work

• Optimization and Multiobjective Control of Time-Discrete Systems
• Scientific Applications: An algorithm for identifying the ergodic subchains and transient states of a stochastic matrix

Cited in

(15)

• Computational discrete time Markov chain with correlated transition probabilities
• Algorithms for Markov stochastic models
• scientific article; zbMATH DE number 7329538 (Why is no real title available?)
• scientific article; zbMATH DE number 4016698 (Why is no real title available?)
• Calculation of higher transitions in a Markov process
• scientific article; zbMATH DE number 494388 (Why is no real title available?)
• An approach for determining the matrix of limiting state probabilities in discrete Markov processes.
• Algorithms for determining the state-time probabilities and the limit matrix in Markov chains
• An effective Markov based approach for calculating the limit matrix in the analytic network process
• About limit matrices of finite-state Markov chains
• Fully polynomial-time computation of maximum likelihood trajectories in Markov chains
• Determining the optimal evolution time for Markov processes with final sequence of states
• Algorithms for determining the transient and differential matrices in finite Markov processes
• Computer aided solving the high-order transition probability matrix of the finite Markov chain
• Optimization of Markov processes with final sequence of states and unitary transition time

This page was built for publication: A dynamic programming approach for finite Markov processes and algorithms for the calculation of the limit matrix in Markov chains

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3111140)