Improved iterative computation of the expected discounted return in Markov and semi-Markov chains

DOI10.1007/BF01919243zbMath0442.90104MaRDI QIDQ3885559

Evan L. Porteus

Publication date: 1980

Published in: Zeitschrift für Operations Research (Search for Journal in Brave)

zbMATH Keywords

convergence; test problems; finite Markov chains; computational experience; semi-Markov chains; iterative computation; expected discounted return; norm reducing extrapolations

Mathematics Subject Classification ID

65K05: Numerical mathematical programming methods

60K15: Markov renewal processes, semi-Markov processes

90C40: Markov and semi-Markov decision processes

Related Items

(Approximate) iterated successive approximations algorithm for sequential decision processes, Computational comparison of policy iteration algorithms for discounted Markov decision processes, On efficiency of linear programming applied to discounted Markovian decision problems, Conditions for characterizing the structure of optimal strategies in infinite-horizon dynamic programs, Second order bounds for Markov decision processes, Generic rank-one corrections for value iteration in Markovian decision problems, Serial and parallel value iteration algorithms for discounted Markov decision processes

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Estimates for finite-stage dynamic programs
• A successive approximation algorithm for an undiscounted Markov decision process
• Discounted Markov games: Generalized policy iteration method
• Contraction mappings underlying undiscounted Markov decision problems
• Dynamic programming, Markov chains, and the method of successive approximations
• A modified dynamic programming method for Markovian decision problems
• Iterative solution of the functional equations of undiscounted Markov renewal programming
• Bounds and Transformations for Discounted Finite Markov Decision Chains
• A decision exclusion algorithm for a class of Markovian Decision Processes
• Discounting, Ergodicity and Convergence for Markov Decision Processes
• A set of successive approximation methods for discounted Markovian decision problems
• Technical Note—Accelerated Computation of the Expected Discounted Return in a Markov Chain
• Modified Policy Iteration Algorithms for Discounted Markov Decision Problems
• Geometric convergence of value-iteration in multichain Markov decision problems
• Markov-Renewal Programming. I: Formulation, Finite Return Models
• Contraction Mappings in the Theory Underlying Dynamic Programming
• On computing the expected discounted return in a markov chain
• Technical Note—On the Asymptotic Convergence Rate of Cost Differences for Markovian Decision Processes
• Technical Note—Undiscounted Markov Renewal Programming Via Modified Successive Approximations
• Some Bounds for Discounted Sequential Decision Processes
• A Generalization of the Additive Correction Methods for the Iterative Solution of Matrix Equations