Contractivity of Bellman operator in risk averse dynamic programming with infinite horizon

DOI10.1016/J.ORL.2023.01.008MaRDI QIDQ6161899zbMATH OpenOpenAlexFDO

Authors Miloš Kopa, Martin Šmíd

Publication date 28 June 2023

Published in Operations Research Letters (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2208.01990

zbMATH Keywords

dynamic programminginfinite horizonrisk aversion

Mathematics Subject Classification ID

Dynamic programming in optimal control and differential games (49L20) Dynamic programming (90C39) Markov and semi-Markov decision processes (90C40)

Abstract: The paper deals with a risk averse dynamic programming problem with infinite horizon. First, the required assumptions are formulated to have the problem well defined. Then the Bellman equation is derived, which may be also seen as a standalone reinforcement learning problem. The fact that the Bellman operator is contraction is proved, guaranteeing convergence of various solution algorithms used for dynamic programming as well as reinforcement learning problems, which we demonstrate on the value iteration algorithm.

Recommendations

• Risk-averse dynamic programming for Markov decision processes
• Abstract dynamic programming
• Elementary results on solutions to the Bellman equation of dynamic programming: existence, uniqueness, and convergence
• Convex dynamic programming with (bounded) recursive utility
• On discounted dynamic programming with unbounded returns

Cites work

• scientific article; zbMATH DE number 1973375 (Why is no real title available?)
• On the dual representation of coherent risk measures
• Reinforcement learning. An introduction
• Risk-averse dynamic programming for Markov decision processes

This page was built for publication: Contractivity of Bellman operator in risk averse dynamic programming with infinite horizon

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6161899)