On the expected total reward with unbounded returns for Markov decision processes

DOI10.1007/S00245-018-9533-6MaRDI QIDQ2198156zbMATH OpenOpenAlexWikidataFDO

Authors F. Dufour, Alexandre Genadot

Publication date 9 September 2020

Published in Applied Mathematics and Optimization (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1712.07874

zbMATH Keywords

expected total rewardMarkov decision processesunbounded returnweak convergence of measure

Mathematics Subject Classification ID

Discrete-time Markov processes on general state spaces (60J05) Markov and semi-Markov decision processes (90C40)

Abstract: We consider a discrete-time Markov decision process with Borel state and action spaces. The performance criterion is to maximize a total expected {utility determined by unbounded return function. It is shown the existence of optimal strategies under general conditions allowing the reward function to be unbounded both from above and below and the action sets available at each step to the decision maker to be not necessarily compact. To deal with unbounded reward functions, a new characterization for the weak convergence of probability measures is derived. Our results are illustrated by examples.

Recommendations

• Arbitrary state semi-Markov decision processes
• scientific article; zbMATH DE number 1536370
• scientific article; zbMATH DE number 934465
• scientific article; zbMATH DE number 1159486
• The finiteness of the reward function and the optimal value function in Markov decision processes

Cites work

• Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal
• Discounted dynamic programming with unbounded returns: application to economic models
• Existence Without Explicit Compactness in Stochastic Dynamic Programming
• Generalised discounting in dynamic programming with unbounded returns
• scientific article; zbMATH DE number 4108536 (Why is no real title available?)
• scientific article; zbMATH DE number 3320878 (Why is no real title available?)
• Markov programming by successive approximations with respect to weighted supremum norms
• Measure theory. Vol. I and II
• On compactness of the space of policies in stochastic dynamic programming
• On discounted dynamic programming with unbounded returns
• On dynamic programming and statistical decision theory
• On dynamic programming: Compactness of the space of policies
• Persistently optimal plans for nonstationary dynamic programming: The topology of weak convergence case
• Stochastic games with unbounded payoffs: applications to robust control in economics
• Stochastic optimal control. The discrete time case
• Unbounded mappings and weak convergence of measures

Cited in

(10)

• Nonatomic total rewards Markov decision processes with multiple criteria
• The finiteness of the reward function and the optimal value function in Markov decision processes
• Constrained discounted stochastic games
• Constrained discounted Markov decision processes with Borel state spaces
• scientific article; zbMATH DE number 3862195 (Why is no real title available?)
• On undiscounted markovian decision processes with compact action spaces
• Arbitrary state semi-Markov decision processes
• scientific article; zbMATH DE number 934465 (Why is no real title available?)
• Constrained Markov decision processes with expected total reward criteria
• Mean-variance optimization in discrete-time decision processes with general utility function

This page was built for publication: On the expected total reward with unbounded returns for Markov decision processes

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2198156)