Optimal control of Markov processes with incomplete state information

DOI10.1016/0022-247X(65)90154-XzbMath0137.35803WikidataQ60164368 ScholiaQ60164368MaRDI QIDQ2521737

Karl Johan Åström

Publication date: 1965

Published in: Journal of Mathematical Analysis and Applications (Search for Journal in Brave)

zbMATH Keywords

probability theory

Related Items (72)

Policy structure for discrete time Markov chain disorder problems ⋮ How to count and guess well: Discrete adaptive filters ⋮ Active inference on discrete state-spaces: a synthesis ⋮ Optimal control of jump-linear gaussian systems† ⋮ PARTIALLY OBSERVABLE MARKOV DECISION PROCESSES AND PERIODIC POLICIES WITH APPLICATIONS ⋮ Performance bounds for discrete-time stochastic optimal control problems† ⋮ A survey of algorithmic methods for partially observed Markov decision processes ⋮ On the average cost optimality equation and the structure of optimal policies for partially observable Markov decision processes ⋮ On the computation of the optimal cost function for discrete time Markov models with partial observations ⋮ Rollout approach to sensor scheduling for remote state estimation under integrity attack ⋮ A unified model of qualitative belief change: a dynamical systems perspective ⋮ Piecewise Linear Approximations for Partially Observable Markov Decision Processes with Finite Horizons ⋮ Markov Decision Processes with Incomplete Information and Semiuniform Feller Transition Probabilities ⋮ Optimal management of stochastic invasion in a metapopulation with Allee effects ⋮ Semi-uniform Feller stochastic kernels ⋮ The free energy principle made simpler but not too simple ⋮ Deep reinforcement learning for adaptive mesh refinement ⋮ Solving zero-sum one-sided partially observable stochastic games ⋮ Equivalent conditions for weak continuity of nonlinear filters ⋮ Monotone control laws for noisy, countable-state Markov chains ⋮ Affect control processes: intelligent affective interaction using a partially observable Markov decision process ⋮ Value Iteration is Optic Composition ⋮ On Markov chains induced by partitioned transition probability matrices ⋮ Partially observable Markov decision model for the treatment of early prostate cancer ⋮ Computation of approximate optimal policies in a partially observed inventory model with rain checks ⋮ Lumpability in compartmental models ⋮ Control: a perspective ⋮ State observation accuracy and finite-memory policy performance ⋮ Unnamed Item ⋮ SWITCHING AND SEQUENCING AVAILABLE THERAPIES SO AS TO MAXIMIZE A PATIENT'S EXPECTED TOTAL LIFETIME ⋮ A survey of decision making and optimization under uncertainty ⋮ Active Inference, Belief Propagation, and the Bethe Approximation ⋮ Unnamed Item ⋮ Uniform Fatou's lemma ⋮ Planning and acting in partially observable stochastic domains ⋮ Non-deterministic weighted automata evaluated over Markov chains ⋮ State estimation for partially observed Markov chains ⋮ On infinite horizon active fault diagnosis for a class of non-linear non-Gaussian systems ⋮ Application of two inequality results for concave functions to a stochastic optimization problem ⋮ Finite-state, discrete-time optimization with randomly varying observation quality ⋮ On the Bellman principle for decision problems with random decision policies ⋮ Probabilistic models for optimizing patients survival rates ⋮ Monotonicity properties for two-action partially observable Markov decision processes on partially ordered spaces ⋮ Politica optima de produccion y control: Un modelo Markoviano ⋮ Separation of estimation and control for decentralized stochastic control systems ⋮ Recursive filters for partially observable finite Markov chains ⋮ Successive approximations in partially observable controlled Markov chains with risk-sensitive average criterion ⋮ Knowledge-based programs as succinct policies for partially observable domains ⋮ Analysis of an identification algorithm arising in the adaptive estimation of Markov chains ⋮ Bottom-up learning of hierarchical models in a class of deterministic pomdp environments ⋮ Optimal stochastic control ⋮ Optimal control of Markov processes with incomplete state-information. II: The convexity of the loss-function ⋮ Remarks on the existence of solutions to the average cost optimality equation in Markov decision processes ⋮ Application of Jensen's inequality to adaptive suboptimal design ⋮ An adaptive automaton controller for discrete-time Markov processes ⋮ Problems of identification and control ⋮ Dual control of an integrator with unknown gain ⋮ Partially observable Markov decision processes with imprecise parameters ⋮ Monitoring machine operations using on-line sensors ⋮ Partially observable environment estimation with uplift inference for reinforcement learning based recommendation ⋮ History-dependent Evaluations in Partially Observable Markov Decision Process ⋮ A tutorial on partially observable Markov decision processes ⋮ Optimizing active surveillance for prostate cancer using partially observable Markov decision processes ⋮ Stratified breast cancer follow-up using a continuous state partially observable Markov decision process ⋮ Active Inference: Demystified and Compared ⋮ Sophisticated Inference ⋮ Transformation of partially observable Markov decision processes into piecewise linear ones ⋮ A survey of solution techniques for the partially observed Markov decision process ⋮ Optimal cost and policy for a Markovian replacement problem ⋮ On the undecidability of probabilistic planning and related stochastic optimization problems ⋮ Recursive estimation of a discrete-time Markov chain ⋮ Optimal sensor scheduling for hidden Markov model state estimation

Cites Work

• On the optimal control of Markov objects
• On the stochastic maximum principle. Fixed time of control
• Optimal stochastic control
• Optimal Training of automatic systems in varying conditions
• Optimum design of digital control systems
• Analytical control design in systems with random properties. III: Optimum control in linear systems. Minimum mean square error
• On the theory of optimal control. Sufficient coordinates
• Conditional Markov Processes
• The Theory of Dynamic Programming as Applied to a Smoothing Problem
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item

This page was built for publication: Optimal control of Markov processes with incomplete state information