Non-deterministic policies in Markovian decision processes

DOI10.1613/JAIR.3175zbMATH Open1210.68080arXiv1401.3871OpenAlexW2159272820WikidataQ113424367 ScholiaQ113424367MaRDI QIDQ3068940FDOQ3068940

Authors: Mahdi Milani Fard, Joelle Pineau 

Publication date: 21 January 2011

Published in: Journal of Artificial Intelligence Research (Search for Journal in Brave)

Abstract: Markovian processes have long been used to model stochastic environments. Reinforcement learning has emerged as a framework to solve sequential planning and decision-making problems in such environments. In recent years, attempts were made to apply methods from reinforcement learning to construct decision support systems for action selection in Markovian environments. Although conventional methods in reinforcement learning have proved to be useful in problems concerning sequential decision-making, they cannot be applied in their current form to decision support systems, such as those in medical domains, as they suggest policies that are often highly prescriptive and leave little room for the users input. Without the ability to provide flexible guidelines, it is unlikely that these methods can gain ground with users of such systems. This paper introduces the new concept of non-deterministic policies to allow more flexibility in the users decision-making process, while constraining decisions to remain near optimal solutions. We provide two algorithms to compute non-deterministic policies in discrete domains. We study the output and running time of these method on a set of synthetic and real-world problems. In an experiment with human subjects, we show that humans assisted by hints based on non-deterministic policies outperform both human-only and computer-only agents in a web navigation task.

Full work available at URL: https://arxiv.org/abs/1401.3871

Recommendations

• Reinforcement learning of non-Markov decision processes
• scientific article
• Multi-objective Markov decision processes for data-driven decision support
• Support of decision-making under uncertainty on the basis of the non-Markov model
• Non-randomized policies for constrained Markov decision processes

zbMATH Keywords

Markov decision processesreinforcement learningdecision-making in complex medical scenarios

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Management decision making, including multiple objectives (90B50) Markov and semi-Markov decision processes (90C40)

Cited In (6)

• Title not available (Why is that?)
• Title not available (Why is that?)
• Nonmyopic Strategic Behavior in the MDP Planning Procedure
• Delayed Nondeterminism in Continuous-Time Markov Decision Processes
• Non-randomized strategies in stochastic decision processes
• Optimal Treatment Regimes: A Review and Empirical Comparison

This page was built for publication: Non-deterministic policies in Markovian decision processes

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3068940)