A survey of multi-objective sequential decision-making

DOI10.1613/JAIR.3987zbMATH Open1364.68323arXiv1402.0590OpenAlexW3103262232MaRDI QIDQ2856473FDOQ2856473

Authors: Diedrik M. Roijers, P. Vamplew, Shimon Whiteson, Richard Dazeley 

Publication date: 29 October 2013

Published in: The Journal of Artificial Intelligence Research (JAIR) (Search for Journal in Brave)

Abstract: Sequential decision-making problems with multiple objectives arise naturally in practice and pose unique challenges for research in decision-theoretic planning and learning, which has largely focused on single-objective settings. This article surveys algorithms designed for sequential decision-making problems with multiple objectives. Though there is a growing body of literature on this subject, little of it makes explicit under what circumstances special methods are needed to solve multi-objective problems. Therefore, we identify three distinct scenarios in which converting such a problem to a single-objective one is impossible, infeasible, or undesirable. Furthermore, we propose a taxonomy that classifies multi-objective methods according to the applicable scenario, the nature of the scalarization function (which projects multi-objective values to scalar ones), and the type of policies considered. We show how these factors determine the nature of an optimal solution, which can be a single policy, a convex hull, or a Pareto front. Using this taxonomy, we survey the literature on multi-objective methods for planning and learning. Finally, we discuss key applications of such methods and outline opportunities for future work.

Full work available at URL: https://arxiv.org/abs/1402.0590

Recommendations

• Some Theory and an Approach to Solving Sequential Multiple-Criteria Decision Problems
• scientific article; zbMATH DE number 3845375
• An approach for simultaneous finding of multiple efficient decisions in multi-objective optimization problems
• A comparison of interactive multiple-objective decision making procedures
• scientific article; zbMATH DE number 1098939
• Solution procedures for multi-objective markov decision processes
• Multi‐objective combinatorial optimization problems: A survey
• Multiple-choice decision making by multicriteria combinatorial optimization
• Multi-attribute sequential decision problem with optimizing and satisficing attributes

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Research exposition (monographs, survey articles) pertaining to computer science (68-02) Problem solving in the context of artificial intelligence (heuristics, search strategies, etc.) (68T20) Decision theory (91B06) Reasoning under uncertainty in the context of artificial intelligence (68T37)

Cited In (26)

• Multi-objective reinforcement learning through continuous Pareto manifold approximation
• Avoiding Negative Side Effects of Autonomous Systems in the Open World
• FlexiBO: A Decoupled Cost-Aware Multi-Objective Optimization Approach for Deep Neural Networks
• A multi-objective approach to the cash management problem
• A Gentle Introduction to Reinforcement Learning
• Reinforcement learning
• Efficient multi-objective reinforcement learning via multiple-gradient descent with iteratively discovered weight-vector sets
• Necessary and sufficient Karush-Kuhn-Tucker conditions for multiobjective Markov chains optimality
• Constrained multiagent Markov decision processes: a taxonomy of problems and algorithms
• Competence-aware systems
• Simple strategies in multi-objective MDPs
• Maximisation of admissible multi-objective heuristics
• Using the Manhattan distance for computing the multiobjective Markov chains problem
• Markov automata with multiple objectives
• A multi-objective approach for PH-graphs with applications to stochastic shortest paths
• Multi-cost bounded tradeoff analysis in MDP
• Computing multiobjective Markov chains handled by the extraproximal method
• Multi-objective dynamic programming with limited precision
• Submodular optimization problems and greedy strategies: a survey
• Reward (Mis)design for autonomous driving
• Multi-objective reinforcement learning using sets of Pareto dominating policies
• Model-based Reinforcement Learning: A Survey
• Challenges of real-world reinforcement learning: definitions, benchmarks and analysis
• Light robustness in the optimization of Markov decision processes with uncertain parameters
• Title not available (Why is that?)
• Multi-objective Markov decision processes for data-driven decision support

This page was built for publication: A survey of multi-objective sequential decision-making

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2856473)