Geometric convergence of value-iteration in multichain Markov decision problems

DOI10.2307/1426774MaRDI QIDQ4187616zbMATH OpenOpenAlexFDO

Authors Awi Federgruen, Paul J. Schweitzer

Publication date 1979

Published in Advances in Applied Probability (Search for Journal in Brave)

Full work available at URL https://doi.org/10.2307/1426774

zbMATH Keywords

Average Cost CriterionConvergence FactorExistence of a Uniform Convergence RateGeometric ConvergenceMarkov Decision ProblemsMudis CountedValue- Iteration Method

Mathematics Subject Classification ID

Minimax problems in mathematical programming (90C47) Rate of convergence, degree of approximation (41A25) Mathematical programming (90C99)

Cited in

(18)

• Nonstationary Markov decision problems with converging parameters
• MARKOV DECISION PROCESSES
• Piecewise affine dynamical models of Petri nets -- application to emergency call centers
• Convergence of iterates in nonlinear Perron-Frobenius theory
• Improved iterative computation of the expected discounted return in Markov and semi-Markov chains
• Guaranteed approximation of Markov chains with applications to multiplexer engineering in ATM networks
• Variational characterizations in Markov decision processes
• Contraction mappings underlying undiscounted Markov decision problems
• A Brouwer fixed-point mapping approach to communicating Markov decision processes
• Illustrated review of convergence conditions of the value iteration algorithm and the rolling horizon procedure for average-cost MDPs
• Criteria for selecting the relaxation factor of the value iteration algorithm for undiscounted Markov and semi-Markov decision processes
• A value iteration method for undiscounted multichain Markov decision processes
• Asymptotic expansions for dynamic programming recursions with general nonnegative matrices
• Contraction mappings underlying undiscounted Markov decision problems. II
• The method of value oriented successive approximations for the average reward Markov decision process
• A value-iteration scheme for undiscounted multichain Markov renewal programs
• Dual bounds on the equilibrium distribution of a finite Markov chain
• Computing transience bounds of emergency call centers: a hierarchical timed Petri net approach

This page was built for publication: Geometric convergence of value-iteration in multichain Markov decision problems

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4187616)