Distributionally robust optimization for sequential decision-making
From MaRDI portal
Abstract: The distributionally robust Markov Decision Process (MDP) approach asks for a distributionally robust policy that achieves the maximal expected total reward under the most adversarial distribution of uncertain parameters. In this paper, we study distributionally robust MDPs where ambiguity sets for the uncertain parameters are of a format that can easily incorporate in its description the uncertainty's generalized moment as well as statistical distance information. In this way, we generalize existing works on distributionally robust MDP with generalized-moment-based and statistical-distance-based ambiguity sets to incorporate information from the former class such as moments and dispersions to the latter class that critically depends on empirical observations of the uncertain parameters. We show that, under this format of ambiguity sets, the resulting distributionally robust MDP remains tractable under mild technical conditions. To be more specific, a distributionally robust policy can be constructed by solving a sequence of one-stage convex optimization subproblems.
Recommendations
- Distributionally robust Markov decision processes
- Distributionally robust partially observable Markov decision process with moment-based ambiguity
- Robust Markov Decision Processes
- Robust Markov Decision Processes with Data-Driven, Distance-Based Ambiguity Sets
- Distributionally Robust Convex Optimization
Cites work
- scientific article; zbMATH DE number 1795843 (Why is no real title available?)
- Ambiguity in portfolio selection
- Ambiguous joint chance constraints under mean and dispersion information
- Approximate Dynamic Programming
- Approximate Value Iteration for Risk-Aware Markov Decision Processes
- Bias and variance approximation in value function estimates
- Coherent measures of risk
- Constructing Risk Measures from Uncertainty Sets
- Constructing uncertainty sets for robust linear optimization
- Data-driven chance constrained stochastic program
- Data-driven risk-averse stochastic optimization with Wasserstein metric
- Distributionally Robust Convex Optimization
- Distributionally Robust Counterpart in Markov Decision Processes
- Distributionally robust Markov decision processes
- Distributionally robust optimization and its tractable approximations
- Distributionally robust optimization under moment uncertainty with application to data-driven problems
- Empirical dynamic programming
- Lectures on modern convex optimization. Analysis, algorithms, and engineering applications
- Likelihood robust optimization for data-driven problems
- Markov Decision Processes with Imprecise Transition Probabilities
- On approximations of data-driven chance constrained programs over Wasserstein balls
- On deterministic reformulations of distributionally robust joint chance constrained optimization problems
- On general minimax theorems
- Optimization of Convex Risk Functions
- Percentile Optimization for Markov Decision Processes with Parameter Uncertainty
- Recovering best statistical guarantees via the empirical divergence-based distributionally robust optimization
- Robust Control of Markov Decision Processes with Uncertain Transition Matrices
- Robust Dynamic Programming
- Robust Markov Decision Processes
- Robust empirical optimization is almost the same as mean-variance optimization
- Robust optimization with ambiguous stochastic constraints under mean and dispersion information
Cited in
(20)- A survey of decision making and optimization under uncertainty
- A dynamical neural network approach for distributionally robust chance-constrained Markov decision process
- Markov decision processes under model uncertainty
- Distributionally Robust Stochastic Dual Dynamic Programming
- Distributionally Robust Reward-Risk Ratio Optimization with Moment Constraints
- Distributionally robust chance-constrained Markov decision processes with random payoff
- Distributionally robust Markov decision processes
- Q-learning for distributionally robust Markov decision processes
- Distributionally robust partially observable Markov decision process with moment-based ambiguity
- An extreme worst-case risk measure by expectile
- An extended robust mathematical model to project the course of COVID-19 epidemic in Iran
- Robust Q-learning algorithm for Markov decision processes under Wasserstein uncertainty
- Wasserstein distributionally robust chance-constrained program with moment information
- Nonconcave stochastic optimal control in finite discrete time under model uncertainty
- Robust MDPs with k-rectangular uncertainty
- Bounding the difference between the values of robust and non-robust Markov decision problems
- Sequential decision-making under uncertainty: a robust MDPs review
- Optimal sequential decision architectures and the robustness of hierarchies and polyarchies
- Robust n-agent heterogeneous investment-consumption game under -maxmin mean-variance-utility criterion
- Robust Markov Decision Processes with Data-Driven, Distance-Based Ambiguity Sets
This page was built for publication: Distributionally robust optimization for sequential decision-making
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5238202)
