Markov decision processes under ambiguity

DOI10.4064/BC122-2MaRDI QIDQ4989141zbMATH OpenOpenAlexFDO

Authors Nicole Bäuerle, Ulrich Rieder

Publication date 20 May 2021

Published in Banach Center Publications (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1907.02347

Mathematics Subject Classification ID

Minimax problems in mathematical programming (90C47) General considerations in statistical decision theory (62C05) Markov and semi-Markov decision processes (90C40)

Abstract: We consider statistical Markov Decision Processes where the decision maker is risk averse against model ambiguity. The latter is given by an unknown parameter which influences the transition law and the cost functions. Risk aversion is either measured by the entropic risk measure or by the Average Value at Risk. We show how to solve these kind of problems using a general minimax theorem. Under some continuity and compactness assumptions we prove the existence of an optimal (deterministic) policy and discuss its computation. We illustrate our results using an example from statistical decision theory.

Recommendations

• Risk-averse dynamic programming for Markov decision processes
• More risk-sensitive Markov decision processes
• Markov decision processes with recursive risk measures
• Markov decision processes with iterated coherent risk measures
• Markov decision processes with average-value-at-risk criteria

Cites work

• scientific article; zbMATH DE number 3320878 (Why is no real title available?)
• scientific article; zbMATH DE number 3357742 (Why is no real title available?)
• scientific article; zbMATH DE number 3103174 (Why is no real title available?)
• A Smooth Model of Decision Making under Ambiguity
• Ambiguity Aversion, Robustness, and the Variational Representation of Preferences
• Ambiguity in asset pricing and portfolio choice: a review of the literature
• Markov decision processes with applications to finance.
• Markov decision processes with average-value-at-risk criteria
• Maxmin expected utility with non-unique prior
• On dynamic programming and statistical decision theory
• On dynamic programming: Compactness of the space of policies
• On general minimax theorems
• Partially Observable Risk-Sensitive Markov Decision Processes
• Random coding strategies for minimum entropy
• Risk-Sensitive Control of Discrete-Time Markov Processes with Infinite Horizon
• Robust Dynamic Programming
• Stochastic finance. An introduction in discrete time

Cited in

(13)

• Optimal investment in ambiguous financial markets with learning
• Distributionally Robust Markov Decision Processes and Their Connection to Risk Measures
• Markov decision processes under model uncertainty
• Ambiguous partially observable Markov decision processes: structural results and applications
• Probabilistic opacity for Markov decision processes
• Markov Decision Processes with Imprecise Transition Probabilities
• Risk filtering and risk-averse control of Markovian systems subject to model uncertainty
• New challenges in the interplay between finance and insurance. Abstracts from the workshop held October 1--6, 2023
• Markov decision processes with iterated coherent risk measures
• Generalizing Markov decision processes to imprecise probabilities
• scientific article; zbMATH DE number 3900544 (Why is no real title available?)
• On modeling risk in Markov decision processes.
• Robust Markov Decision Processes with Data-Driven, Distance-Based Ambiguity Sets

This page was built for publication: Markov decision processes under ambiguity

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4989141)