Uniform value in dynamic programming

DOI10.4171/JEMS/254MaRDI QIDQ621850zbMATH OpenFDO

Authors Jérôme Renault

Publication date 28 January 2011

Published in Journal of the European Mathematical Society (JEMS) (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/0803.2758

zbMATH Keywords

uniform value, dynamic programming, Markov decision processes, limit value, Blackwell optimality, average payoffs, long-run values, precompact state space, non expansive correspondence

Mathematics Subject Classification ID

Dynamic programming (90C39) Markov and semi-Markov decision processes (90C40)

Abstract: We consider dynamic programming problems with a large time horizon, and give sufficient conditions for the existence of the uniform value. As a consequence, we obtain an existence result when the state space is precompact, payoffs are uniformly continuous and the transition correspondence is non expansive. In the same spirit, we give an existence result for the limit value. We also apply our results to Markov decision processes and obtain a few generalizations of existing results.

Recommendations

• General limit value in dynamic programming
• Asymptotic properties in dynamic programming
• Low discounting and the upper long-run average value in dynamic programming
• Strong uniform value in gambling houses and partially observable Markov decision processes
• A Uniform Tauberian Theorem in Dynamic Programming

Cited in

(31)

• Ergodicity conditions for zero-sum games
• A zero-sum stochastic game with compact action sets and no asymptotic value
• On values of repeated games with signals
• Existence of the uniform value in zero-sum repeated games with a more informed controller
• Asymptotic properties of optimal trajectories in dynamic programming
• Existence of asymptotic values for nonexpansive stochastic control systems
• Bounded variation of \(\{V_ n\}\) and its limit
• LP based upper and lower bounds for Cesàro and Abel limits of the optimal values in problems of control of stochastic discrete time systems
• Representation formulas for limit values of long run stochastic optimal controls
• Commutative Stochastic Games
• Acyclic Gambling Games
• Tauberian theorem for value functions
• Strong uniform value in gambling houses and partially observable Markov decision processes
• Finitely additive dynamic programming
• Definable zero-sum stochastic games
• Long-term values in Markov decision processes and repeated games, and a new distance for probability spaces
• Linear programming formulations of deterministic infinite horizon optimal control problems in discrete time
• Generic uniqueness of the bias vector of finite zero-sum stochastic games with perfect information
• Asymptotic control for a class of piecewise deterministic Markov processes associated to temperate viruses
• A survey of average cost problems in deterministic discrete-time control systems
• Limit value for optimal control with general means
• An accretive operator approach to ergodic zero-sum stochastic games
• Finite-memory strategies in POMDPs with long-run average objectives
• A Tauberian theorem for nonexpansive operators and applications to zero-sum stochastic games
• General limit value in dynamic programming
• History-dependent evaluations in partially observable Markov decision process
• On representation formulas for long run averaging optimal control problem
• Stochastic games
• Vanishing Discount Limit and Nonexpansive Optimal Control and Differential Games
• A game theory approach to the existence and uniqueness of nonlinear Perron-Frobenius eigenvectors
• Communicating zero-sum product stochastic games

This page was built for publication: Uniform value in dynamic programming

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q621850)