A survey of algorithmic methods for partially observed Markov decision processes

DOI10.1007/BF02055574zbMath0717.90086MaRDI QIDQ2638960

William S. Lovejoy

Publication date: 1991

Published in: Annals of Operations Research (Search for Journal in Brave)

zbMATH Keywords

surveyincomplete informationpartially observed Markov decision processfinite and infinite horizonsapproximation methodologies

Mathematics Subject Classification ID

Markov and semi-Markov decision processes (90C40) Research exposition (monographs, survey articles) pertaining to operations research and mathematical programming (90-02) Computational methods for problems pertaining to operations research and mathematical programming (90-08)

Related Items

A two-state partially observable Markov decision process with three actions, Dynamic Pricing and Learning with Finite Inventories, Model Checking Linear-Time Properties of Probabilistic Systems, Admission Control Policies in a Finite Capacity Geo/Geo/1 Queue Under Partial State Observations, Planning for multiple measurement channels in a continuous-state POMDP, An efficient heuristic for a partially observable Markov decision process of machine replacement, Control limits for two-state partially observable Markov decision processes, Asymptotically optimal Bayesian sequential change detection and identification rules, A unified model of qualitative belief change: a dynamical systems perspective, Cops and invisible robbers: the cost of drunkenness, Dynamic Learning and Decision Making via Basis Weight Vectors, BOUNDED-PARAMETER PARTIALLY OBSERVABLE MARKOV DECISION PROCESSES: FRAMEWORK AND ALGORITHM, Abstraction and approximate decision-theoretic planning., Finding optimal memoryless policies of POMDPs under the expected average reward criterion, Affect control processes: intelligent affective interaction using a partially observable Markov decision process, Computation of approximate optimal policies in a partially observed inventory model with rain checks, A unified framework for stochastic optimization, SLAP: specification logic of actions with probability, State observation accuracy and finite-memory policy performance, Markov limid processes for representing and solving renewal problems, Optimal Online Learning for Nonlinear Belief Models Using Discrete Priors, An Approximation Approach for Response-Adaptive Clinical Trial Design, An integrated approach to solving influence diagrams and finite-horizon partially observable decision processes, A survey of decision making and optimization under uncertainty, Unnamed Item, Markov-Entscheidungs-Prozesse mit abhängigen Aktionen für optimale Reparaturmaßnahmen bei unvollständiger Information. (Markov decision processes with dependent actions for optimal repair policies under incomplete information), Modelling of hydrological persistence for hidden state Markov decision processes, Planning and acting in partially observable stochastic domains, Performance prediction of an unmanned airborne vehicle multi-agent system, Heuristic anytime approaches to stochastic decision processes, Partially observable Markov decision process approximations for adaptive sensing, Probabilistic Acceptors for Languages over Infinite Words, Optimal condition based maintenance with imperfect information and the proportional hazards model, Undiscounted Markov decision chains with partial information; an algorithm for computing a locally optimal periodic policy, A simple suboptimal algorithm for system maintance under partial observability, Selecting a quality control attribute sample: An information-economics method, Partially observable Markov decision processes with imprecise parameters, A Fenchel-Moreau-Rockafellar type theorem on the Kantorovich-Wasserstein space with applications in partially observable Markov decision processes, Predictive control of discrete time stochastic nonlinear state space dynamical systems: a particle nonparametric approach, Influence of modeling structure in probabilistic sequential decision problems, A tutorial on partially observable Markov decision processes, Optimizing active surveillance for prostate cancer using partially observable Markov decision processes, Stochastic dynamic programming with factored representations, Stratified breast cancer follow-up using a continuous state partially observable Markov decision process, A simulation-based approach to stochastic dynamic programming, Optimal decisions in stochastic graphs with uncorrelated and correlated edge weights, A leader-follower partially observed, multiobjective Markov game, Value of information for a leader-follower partially observed Markov game, Optimal sensor scheduling for hidden Markov model state estimation

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• A course in triangulations for solving equations with deformations
• Monotone control laws for noisy, countable-state Markov chains
• Contraction mappings underlying undiscounted Markov decision problems
• Optimal control of Markov processes with incomplete state information
• Optimal control of partially observable Markovian systems
• Conditions for the Existence of Planning Horizons
• The Optimal Search for a Moving Target When the Search Path Is Constrained
• Some Monotonicity Results for Partially Observed Markov Decision Processes
• Optimal Infinite-Horizon Undiscounted Control of Finite Probabilistic Systems
• State of the Art—A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms
• Computationally Feasible Bounds for Partially Observed Markov Decision Processes
• Discounting, Ergodicity and Convergence for Markov Decision Processes
• Markovian Deterioration with Uncertain Information
• The Optimal Control of Partially Observable Markov Processes over the Infinite Horizon: Discounted Costs
• OPTIMAL CONTROL FOR PARTIALLY OBSERVABLE MARKOV DECISION PROCESSES OVER AN INFINITE HORIZON
• Optimal control-limit strategies for a partially observed replacement problem†
• Finite-Memory Suboptimal Design for Partially Observed Markov Decision Processes
• The Optimal Control of Partially Observable Markov Processes over a Finite Horizon
• Solution Procedures for Partially Observed Markov Decision Processes
• Discounted Dynamic Programming
• Letter to the Editor—A Test for Suboptimal Actions in Markovian Decision Problems
• Convex Analysis
• Quality Control under Markovian Deterioration