Affine Monotonic and Risk-Sensitive Models in Dynamic Programming
From MaRDI portal
Publication:5228296
DOI10.1109/TAC.2019.2896049zbMATH Open1482.90237arXiv1608.01393OpenAlexW2963213435MaRDI QIDQ5228296FDOQ5228296
Authors: Dimitri P. Bertsekas 
Publication date: 12 August 2019
Published in: IEEE Transactions on Automatic Control (Search for Journal in Brave)
Abstract: In this paper we consider a broad class of infinite horizon discrete-time optimal control models that involve a nonnegative cost function and an affine mapping in their dynamic programming equation. They include as special cases classical models such as stochastic undiscounted nonnegative cost problems, stochastic multiplicative cost problems, and risk-sensitive problems with exponential cost. We focus on the case where the state space is finite and the control space has some compactness properties. We assume that the affine mapping has a semicontractive character, whereby for some policies it is a contraction, while for others it is not. In one line of analysis, we impose assumptions that guarantee that the latter policies cannot be optimal. Under these assumptions, we prove strong results that resemble those for discounted Markovian decision problems, such as the uniqueness of solution of Bellman's equation, and the validity of forms of value and policy iteration. In the absence of these assumptions, the results are weaker and unusual in character: the optimal cost function need not be a solution of Bellman's equation, and an optimal policy may not be found by value or policy iteration. Instead the optimal cost function over just the contractive policies solves Bellman's equation, and can be computed by a variety of algorithms.
Full work available at URL: https://arxiv.org/abs/1608.01393
Cited In (2)
This page was built for publication: Affine Monotonic and Risk-Sensitive Models in Dynamic Programming
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5228296)
