Contraction mappings underlying undiscounted Markov decision problems

DOI10.1016/0022-247X(78)90174-9zbMath0388.90084OpenAlexW4205234699MaRDI QIDQ1250794

Paul J. Schweitzer, Awi Federgruen, Henk C. Tijms

Publication date: 1978

Published in: Journal of Mathematical Analysis and Applications (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1016/0022-247x(78)90174-9

Mathematics Subject Classification ID

Minimax problems in mathematical programming (90C47) Markov and semi-Markov decision processes (90C40)

Related Items

A unified approach to adaptive control of average reward Markov decision processes, Contraction mappings underlying undiscounted Markov decision problems. II, A survey of algorithmic methods for partially observed Markov decision processes, A Brouwer fixed-point mapping approach to communicating Markov decision processes, Nonstationary Markov decision problems with converging parameters, Conditions for characterizing the structure of optimal strategies in infinite-horizon dynamic programs, Illustrated review of convergence conditions of the value iteration algorithm and the rolling horizon procedure for average-cost MDPs, Iterative algorithms for solving undiscounted bellman equations, Stationary equilibria in cyclic games: search and structure, Unnamed Item, A Fixed Point Approach to Undiscounted Markov Renewal Programs, Improved iterative computation of the expected discounted return in Markov and semi-Markov chains, Denumerable state nonhomogeneous Markov decision processes, Finite state approximation algorithms for average cost denumerable state Markov decision processes

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Variational characterizations in Markov decision processes
• Solving Markovian decision processes by successive elimination of variables
• Exponential convergence of products of stochastic matrices
• Dynamic programming, Markov chains, and the method of successive approximations
• A modified dynamic programming method for Markovian decision problems
• Iterative solution of the functional equations of undiscounted Markov renewal programming
• Finite state Markovian decision processes
• An iterative method for approximating average cost optimal (s,S) inventory policies
• Note—A Test for Nonoptimal Actions in Undiscounted Finite Markov Decision Chains
• Discounting, Ergodicity and Convergence for Markov Decision Processes
• The Functional Equations of Undiscounted Markov Renewal Programming
• Geometric convergence of value-iteration in multichain Markov decision problems
• The Asymptotic Behavior of Undiscounted Value Iteration in Markov Decision Problems
• Optimal decision procedures for finite Markov chains. Part II: Communicating systems
• Markov-Renewal Programming. I: Formulation, Finite Return Models
• On the Iterative Method of Dynamic Programming on a Finite Space Discrete Time Markov Process
• Contraction Mappings in the Theory Underlying Dynamic Programming
• Étude asymptotique des systèmes markoviens à commande
• Letter to the Editor—A Test for Suboptimal Actions in Markovian Decision Problems
• On Finding the Maximal Gain for Markov Decision Processes
• Multichain Markov Renewal Programs
• Technical Note—Bounds on the Gain of a Markov Decision Process
• Some Bounds for Discounted Sequential Decision Processes
• Generalization of White's Method of Successive Approximations to Periodic Markovian Decision Processes