Distributionally Robust Counterpart in Markov Decision Processes

DOI10.1109/TAC.2015.2495174MaRDI QIDQ2980400zbMATH OpenFDO

Authors Pengqian Yu, Huan Xu

Publication date 3 May 2017

Published in IEEE Transactions on Automatic Control (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1501.07418

Mathematics Subject Classification ID

Markov and semi-Markov decision processes (90C40)

Abstract: This paper studies Markov Decision Processes under parameter uncertainty. We adapt the distributionally robust optimization framework, and assume that the uncertain parameters are random variables following an unknown distribution, and seeks the strategy which maximizes the expected performance under the most adversarial distribution. In particular, we generalize previous study cite{xu2012distributionally} which concentrates on distribution sets with very special structure to much more generic class of distribution sets, and show that the optimal strategy can be obtained efficiently under mild technical condition. This significantly extends the applicability of distributionally robust MDP to incorporate probabilistic information of uncertainty in a more flexible way.

Cited in

(17)

• Risk-averse model predictive control
• A dynamic game approach to distributionally robust safety specifications for stochastic systems
• Robust analysis of discounted Markov decision processes with uncertain transition probabilities
• Frameworks and results in distributionally robust optimization
• A survey of decision making and optimization under uncertainty
• An active-set strategy to solve Markov decision processes with good-deal risk measure
• Poisoning finite-horizon Markov decision processes at design time
• Robust Markov Decision Processes with Data-Driven, Distance-Based Ambiguity Sets
• Distributionally robust partially observable Markov decision process with moment-based ambiguity
• Robust Markov Decision Processes
• Extended Laplace principle for empirical measures of a Markov chain
• Infinite Horizon Average Cost Dynamic Programming Subject to Total Variation Distance Ambiguity
• Distributionally robust optimization for sequential decision-making
• Distributionally Robust Markov Decision Processes and Their Connection to Risk Measures
• A Bayesian approach to data-driven multi-stage stochastic optimization
• Set-based value operators for non-stationary and uncertain Markov decision processes
• Sequential decision-making under uncertainty: a robust MDPs review

This page was built for publication: Distributionally Robust Counterpart in Markov Decision Processes

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2980400)