Markov decision processes under ambiguity

DOI10.4064/BC122-2zbMATH Open1460.90199arXiv1907.02347OpenAlexW2954037867MaRDI QIDQ4989141FDOQ4989141

Authors: Nicole Bäuerle, Ulrich Rieder 

Publication date: 20 May 2021

Published in: Banach Center Publications (Search for Journal in Brave)

Abstract: We consider statistical Markov Decision Processes where the decision maker is risk averse against model ambiguity. The latter is given by an unknown parameter which influences the transition law and the cost functions. Risk aversion is either measured by the entropic risk measure or by the Average Value at Risk. We show how to solve these kind of problems using a general minimax theorem. Under some continuity and compactness assumptions we prove the existence of an optimal (deterministic) policy and discuss its computation. We illustrate our results using an example from statistical decision theory.

Full work available at URL: https://arxiv.org/abs/1907.02347

Recommendations

• Risk-averse dynamic programming for Markov decision processes
• More risk-sensitive Markov decision processes
• Markov decision processes with recursive risk measures
• Markov decision processes with iterated coherent risk measures
• Markov decision processes with average-value-at-risk criteria

Mathematics Subject Classification ID

Minimax problems in mathematical programming (90C47) General considerations in statistical decision theory (62C05) Markov and semi-Markov decision processes (90C40)

Cites Work

• Title not available (Why is that?)
• Title not available (Why is that?)
• Maxmin expected utility with non-unique prior
• Ambiguity Aversion, Robustness, and the Variational Representation of Preferences
• A Smooth Model of Decision Making under Ambiguity
• Stochastic finance. An introduction in discrete time
• On general minimax theorems
• Robust Dynamic Programming
• Risk-Sensitive Control of Discrete-Time Markov Processes with Infinite Horizon
• Markov decision processes with applications to finance.
• On dynamic programming: Compactness of the space of policies
• Title not available (Why is that?)
• Ambiguity in asset pricing and portfolio choice: a review of the literature
• Markov decision processes with average-value-at-risk criteria
• Partially Observable Risk-Sensitive Markov Decision Processes
• On dynamic programming and statistical decision theory
• Random coding strategies for minimum entropy

Cited In (13)

• Optimal investment in ambiguous financial markets with learning
• Distributionally Robust Markov Decision Processes and Their Connection to Risk Measures
• Markov decision processes under model uncertainty
• Ambiguous partially observable Markov decision processes: structural results and applications
• Probabilistic opacity for Markov decision processes
• Markov Decision Processes with Imprecise Transition Probabilities
• Risk filtering and risk-averse control of Markovian systems subject to model uncertainty
• New challenges in the interplay between finance and insurance. Abstracts from the workshop held October 1--6, 2023
• Markov decision processes with iterated coherent risk measures
• Generalizing Markov decision processes to imprecise probabilities
• Title not available (Why is that?)
• On modeling risk in Markov decision processes.
• Robust Markov Decision Processes with Data-Driven, Distance-Based Ambiguity Sets

This page was built for publication: Markov decision processes under ambiguity

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4989141)