Markov decision processes with incomplete information and semiuniform Feller transition probabilities
From MaRDI portal
Publication:5097394
Abstract: This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important feature of these models is that their classic reduction to Completely Observable Markov Decision Processes with belief states preserves semi-uniform Feller continuity of transition probabilities. Under mild assumptions on cost functions, optimal policies exist, optimality equations hold, and value iterations converge to optimal values for these models. In particular, for Partially Observable Markov Decision Processes the results of this paper imply new and generalize several known sufficient conditions on transition and observation probabilities for weak continuity of transition probabilities for Markov Decision Processes with belief states, the existence of optimal policies, validity of optimality equations defining optimal policies, and convergence of value iterations to optimal values.
Recommendations
- Partially observable total-cost Markov decision processes with weakly continuous transition probabilities
- scientific article; zbMATH DE number 4156234
- Average cost Markov decision processes with semi-uniform Feller transition probabilities
- Structural properties for contracting state partially observable Markov decision processes
- scientific article; zbMATH DE number 9756
Cites work
- scientific article; zbMATH DE number 3664132 (Why is no real title available?)
- scientific article; zbMATH DE number 3692372 (Why is no real title available?)
- scientific article; zbMATH DE number 837313 (Why is no real title available?)
- scientific article; zbMATH DE number 3223255 (Why is no real title available?)
- scientific article; zbMATH DE number 3238721 (Why is no real title available?)
- scientific article; zbMATH DE number 3245885 (Why is no real title available?)
- scientific article; zbMATH DE number 3274494 (Why is no real title available?)
- Adaptive Markov control processes
- Average Optimality in Dynamic Programming with General State Space
- Average cost Markov decision processes with semi-uniform Feller transition probabilities
- Average cost Markov decision processes with weakly continuous transition probabilities
- Average optimality in dynamic programming on Borel spaces -- unbounded costs and controls
- Bayesian dynamic programming
- Berge's theorem for noncompact image sets
- Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal
- Convergence of probability measures and Markov decision models with incomplete information
- Fatou's lemma for weakly converging probabilities
- Fatou's lemma in its classical form and Lebesgue's convergence theorems for varying measures with applications to Markov decision processes
- Incomplete information in Markovian decision models
- MDPs with setwise continuous transition probabilities
- Markov decision processes with applications to finance.
- Measure theory. Vol. I and II
- On compactness of the space of policies in stochastic dynamic programming
- On convergence of value iteration for a class of total cost Markov decision processes
- On dynamic programming: Compactness of the space of policies
- On essential information in sequential decision processes
- Optimal Infinite-Horizon Undiscounted Control of Finite Probabilistic Systems
- Optimal control of Markov processes with incomplete state information
- Optimal control of partially observable Markovian systems
- Partially observable total-cost Markov decision processes with weakly continuous transition probabilities
- Real Analysis and Probability
- Reduction of a Controlled Markov Model with Incomplete Data to a Problem with Complete Information in the Case of Borel State and Control Space
- State of the Art—A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms
- Stochastic optimal control. The discrete time case
- The Complexity of Markov Decision Processes
- The Optimal Control of Partially Observable Markov Processes over a Finite Horizon
- The Optimal Control of Partially Observable Markov Processes over the Infinite Horizon: Discounted Costs
- The critical discount factor for finite Markovian decision processes with an absorbing set
- Uniform Fatou's lemma
- Weak Feller property of non-linear filters
Cited in
(12)- Partially observable total-cost Markov decision processes with weakly continuous transition probabilities
- Markov Decision Processes with Imprecise Transition Probabilities
- scientific article; zbMATH DE number 176287 (Why is no real title available?)
- scientific article; zbMATH DE number 4156234 (Why is no real title available?)
- Structural properties for contracting state partially observable Markov decision processes
- Convergence of probability measures and Markov decision models with incomplete information
- Optimality conditions for partially observable Markov decision processes
- Formalization of methods for the development of autonomous artificial intelligence systems
- Another look at partially observed optimal stochastic control: existence, ergodicity, and approximations without belief-reduction
- Equivalent conditions for weak continuity of nonlinear filters
- Semi-uniform Feller stochastic kernels
- Information-theoretic multi-time-scale partially observable systems with inspiration from leukemia treatment
This page was built for publication: Markov decision processes with incomplete information and semiuniform Feller transition probabilities
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5097394)
