Distributionally robust optimization for sequential decision-making
From MaRDI portal
Abstract: The distributionally robust Markov Decision Process (MDP) approach asks for a distributionally robust policy that achieves the maximal expected total reward under the most adversarial distribution of uncertain parameters. In this paper, we study distributionally robust MDPs where ambiguity sets for the uncertain parameters are of a format that can easily incorporate in its description the uncertainty's generalized moment as well as statistical distance information. In this way, we generalize existing works on distributionally robust MDP with generalized-moment-based and statistical-distance-based ambiguity sets to incorporate information from the former class such as moments and dispersions to the latter class that critically depends on empirical observations of the uncertain parameters. We show that, under this format of ambiguity sets, the resulting distributionally robust MDP remains tractable under mild technical conditions. To be more specific, a distributionally robust policy can be constructed by solving a sequence of one-stage convex optimization subproblems.
Recommendations
- Distributionally robust Markov decision processes
- Distributionally robust partially observable Markov decision process with moment-based ambiguity
- Robust Markov Decision Processes
- Robust Markov Decision Processes with Data-Driven, Distance-Based Ambiguity Sets
- Distributionally Robust Convex Optimization
Cites work
- Ambiguity in portfolio selection
- Ambiguous joint chance constraints under mean and dispersion information
- Approximate Dynamic Programming
- Approximate Value Iteration for Risk-Aware Markov Decision Processes
- Bias and variance approximation in value function estimates
- Coherent measures of risk
- Constructing Risk Measures from Uncertainty Sets
- Constructing uncertainty sets for robust linear optimization
- Data-driven chance constrained stochastic program
- Data-driven risk-averse stochastic optimization with Wasserstein metric
- Distributionally Robust Convex Optimization
- Distributionally Robust Counterpart in Markov Decision Processes
- Distributionally robust Markov decision processes
- Distributionally robust optimization and its tractable approximations
- Distributionally robust optimization under moment uncertainty with application to data-driven problems
- Empirical dynamic programming
- scientific article; zbMATH DE number 1795843 (Why is no real title available?)
- Lectures on modern convex optimization. Analysis, algorithms, and engineering applications
- Likelihood robust optimization for data-driven problems
- Markov Decision Processes with Imprecise Transition Probabilities
- On approximations of data-driven chance constrained programs over Wasserstein balls
- On deterministic reformulations of distributionally robust joint chance constrained optimization problems
- On general minimax theorems
- Optimization of Convex Risk Functions
- Percentile Optimization for Markov Decision Processes with Parameter Uncertainty
- Recovering best statistical guarantees via the empirical divergence-based distributionally robust optimization
- Robust Control of Markov Decision Processes with Uncertain Transition Matrices
- Robust Dynamic Programming
- Robust empirical optimization is almost the same as mean-variance optimization
- Robust Markov Decision Processes
- Robust optimization with ambiguous stochastic constraints under mean and dispersion information
Cited in
(23)- Distributionally robust optimal control and MDP modeling
- A survey of decision making and optimization under uncertainty
- Optimal sequential decision architectures and the robustness of hierarchies and polyarchies
- Robust MDPs with k-rectangular uncertainty
- Distributionally robust Markov decision processes
- Distributionally Robust Stochastic Dual Dynamic Programming
- Robust Markov Decision Processes with Data-Driven, Distance-Based Ambiguity Sets
- Distributionally robust partially observable Markov decision process with moment-based ambiguity
- Q-learning for distributionally robust Markov decision processes
- Distributionally Robust Reward-Risk Ratio Optimization with Moment Constraints
- Markov decision processes under model uncertainty
- Wasserstein distributionally robust chance-constrained program with moment information
- A dynamical neural network approach for distributionally robust chance-constrained Markov decision process
- Distributionally robust chance-constrained Markov decision processes with random payoff
- An extended robust mathematical model to project the course of COVID-19 epidemic in Iran
- Robust Q-learning algorithm for Markov decision processes under Wasserstein uncertainty
- Nonconcave stochastic optimal control in finite discrete time under model uncertainty
- Bounding the difference between the values of robust and non-robust Markov decision problems
- Sequential decision-making under uncertainty: a robust MDPs review
- Robust n-agent heterogeneous investment-consumption game under -maxmin mean-variance-utility criterion
- An extreme worst-case risk measure by expectile
- Robust probabilistic inference via a constrained transport metric (with discussion)
- Robust competitive ratio for deterministic monopoly pricing
This page was built for publication: Distributionally robust optimization for sequential decision-making
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5238202)
