Uniform value in dynamic programming

DOI10.4171/JEMS/254zbMATH Open1229.90253arXiv0803.2758MaRDI QIDQ621850FDOQ621850

Authors: Jérôme Renault 

Publication date: 28 January 2011

Published in: Journal of the European Mathematical Society (JEMS) (Search for Journal in Brave)

Abstract: We consider dynamic programming problems with a large time horizon, and give sufficient conditions for the existence of the uniform value. As a consequence, we obtain an existence result when the state space is precompact, payoffs are uniformly continuous and the transition correspondence is non expansive. In the same spirit, we give an existence result for the limit value. We also apply our results to Markov decision processes and obtain a few generalizations of existing results.

Full work available at URL: https://arxiv.org/abs/0803.2758

Recommendations

• General limit value in dynamic programming
• Asymptotic properties in dynamic programming
• Low discounting and the upper long-run average value in dynamic programming
• Strong uniform value in gambling houses and partially observable Markov decision processes
• A Uniform Tauberian Theorem in Dynamic Programming

zbMATH Keywords

uniform value, dynamic programming, Markov decision processes, limit value, Blackwell optimality, average payoffs, long-run values, precompact state space, non expansive correspondence

Mathematics Subject Classification ID

Dynamic programming (90C39) Markov and semi-Markov decision processes (90C40)

Cited In (30)

• Ergodicity conditions for zero-sum games
• Definable Zero-Sum Stochastic Games
• A zero-sum stochastic game with compact action sets and no asymptotic value
• On values of repeated games with signals
• Finite-Memory Strategies in POMDPs with Long-Run Average Objectives
• Existence of the uniform value in zero-sum repeated games with a more informed controller
• Asymptotic properties of optimal trajectories in dynamic programming
• Asymptotic Control for a Class of Piecewise Deterministic Markov Processes Associated to Temperate Viruses
• Existence of asymptotic values for nonexpansive stochastic control systems
• Bounded variation of \(\{V_ n\}\) and its limit
• LP based upper and lower bounds for Cesàro and Abel limits of the optimal values in problems of control of stochastic discrete time systems
• Commutative Stochastic Games
• Acyclic Gambling Games
• Tauberian theorem for value functions
• Strong uniform value in gambling houses and partially observable Markov decision processes
• History-dependent Evaluations in Partially Observable Markov Decision Process
• Finitely additive dynamic programming
• Linear programming formulations of deterministic infinite horizon optimal control problems in discrete time
• Generic uniqueness of the bias vector of finite zero-sum stochastic games with perfect information
• A survey of average cost problems in deterministic discrete-time control systems
• Limit value for optimal control with general means
• An accretive operator approach to ergodic zero-sum stochastic games
• Stochastic Games
• A Tauberian theorem for nonexpansive operators and applications to zero-sum stochastic games
• General limit value in dynamic programming
• Vanishing Discount Limit and Nonexpansive Optimal Control and Differential Games
• On representation formulas for long run averaging optimal control problem
• A game theory approach to the existence and uniqueness of nonlinear Perron-Frobenius eigenvectors
• Representation Formulas for Limit Values of Long Run Stochastic Optimal Controls
• Communicating zero-sum product stochastic games

This page was built for publication: Uniform value in dynamic programming

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q621850)