Markov decision processes with incomplete information and semiuniform Feller transition probabilities
From MaRDI portal
(Redirected from Publication:5097394)
Abstract: This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important feature of these models is that their classic reduction to Completely Observable Markov Decision Processes with belief states preserves semi-uniform Feller continuity of transition probabilities. Under mild assumptions on cost functions, optimal policies exist, optimality equations hold, and value iterations converge to optimal values for these models. In particular, for Partially Observable Markov Decision Processes the results of this paper imply new and generalize several known sufficient conditions on transition and observation probabilities for weak continuity of transition probabilities for Markov Decision Processes with belief states, the existence of optimal policies, validity of optimality equations defining optimal policies, and convergence of value iterations to optimal values.
Recommendations
- Partially observable total-cost Markov decision processes with weakly continuous transition probabilities
- scientific article; zbMATH DE number 4156234
- Average cost Markov decision processes with semi-uniform Feller transition probabilities
- Structural properties for contracting state partially observable Markov decision processes
- scientific article; zbMATH DE number 9756
Cites work
- Adaptive Markov control processes
- Average cost Markov decision processes with semi-uniform Feller transition probabilities
- Average cost Markov decision processes with weakly continuous transition probabilities
- Average optimality in dynamic programming on Borel spaces -- unbounded costs and controls
- Average Optimality in Dynamic Programming with General State Space
- Bayesian dynamic programming
- Berge's theorem for noncompact image sets
- Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal
- Convergence of probability measures and Markov decision models with incomplete information
- Fatou's lemma for weakly converging probabilities
- Fatou's lemma in its classical form and Lebesgue's convergence theorems for varying measures with applications to Markov decision processes
- scientific article; zbMATH DE number 3664132 (Why is no real title available?)
- scientific article; zbMATH DE number 3692372 (Why is no real title available?)
- scientific article; zbMATH DE number 837313 (Why is no real title available?)
- scientific article; zbMATH DE number 3223255 (Why is no real title available?)
- scientific article; zbMATH DE number 3238721 (Why is no real title available?)
- scientific article; zbMATH DE number 3245885 (Why is no real title available?)
- scientific article; zbMATH DE number 3274494 (Why is no real title available?)
- Incomplete information in Markovian decision models
- Markov decision processes with applications to finance.
- MDPs with setwise continuous transition probabilities
- Measure theory. Vol. I and II
- On compactness of the space of policies in stochastic dynamic programming
- On convergence of value iteration for a class of total cost Markov decision processes
- On dynamic programming: Compactness of the space of policies
- On essential information in sequential decision processes
- Optimal control of Markov processes with incomplete state information
- Optimal control of partially observable Markovian systems
- Optimal Infinite-Horizon Undiscounted Control of Finite Probabilistic Systems
- Partially observable total-cost Markov decision processes with weakly continuous transition probabilities
- Real Analysis and Probability
- Reduction of a Controlled Markov Model with Incomplete Data to a Problem with Complete Information in the Case of Borel State and Control Space
- State of the Art—A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms
- Stochastic optimal control. The discrete time case
- The Complexity of Markov Decision Processes
- The critical discount factor for finite Markovian decision processes with an absorbing set
- The Optimal Control of Partially Observable Markov Processes over a Finite Horizon
- The Optimal Control of Partially Observable Markov Processes over the Infinite Horizon: Discounted Costs
- Uniform Fatou's lemma
- Weak Feller property of non-linear filters
Cited in
(17)- Structural properties for contracting state partially observable Markov decision processes
- Partially observable total-cost Markov decision processes with weakly continuous transition probabilities
- Optimality conditions for partially observable Markov decision processes
- scientific article; zbMATH DE number 4156234 (Why is no real title available?)
- scientific article; zbMATH DE number 176287 (Why is no real title available?)
- Markov Decision Processes with Imprecise Transition Probabilities
- Convergence of probability measures and Markov decision models with incomplete information
- scientific article; zbMATH DE number 4195739 (Why is no real title available?)
- Formalization of methods for the development of autonomous artificial intelligence systems
- Semi-uniform Feller stochastic kernels
- Equivalent conditions for weak continuity of nonlinear filters
- Information-theoretic multi-time-scale partially observable systems with inspiration from leukemia treatment
- Another look at partially observed optimal stochastic control: existence, ergodicity, and approximations without belief-reduction
- Partially observable discrete-time stochastic games under risk probability criterion
- Sequential convergence on the space of Borel measures
- Continuity of filters for discrete-time control problems defined by explicit equations
- Finite horizon partially observable semi-Markov decision processes under risk probability criteria
This page was built for publication: Markov decision processes with incomplete information and semiuniform Feller transition probabilities
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5097394)
