Semi-infinite Markov decision processes
This paper studies Markov Decision Processes (MDPs) with finite state and countable action sets. It deals with two major criteria for MDPs: the total discounted rewards and average rewards per unit time. For the total discounted criteria, the authors study the linear programming approach. In particular, they develop a semi-infinite linear program for the MDP and show that it can be approximated by finite linear programs associated with finite state and action MDPs, which are truncated versions of the original MDPs. The authors also investigate the question when the supremum of the objective criterion over the class of deterministic (or stationary, according to another terminology) policies is equal to the supremum over the class of all policies. Referee's remark: (i) Theorem 3.1, which states that the supremum of the total discounted rewards over all policies is equal to the supremum over deterministic policies, is a particular case of Theorem 5 (b) in \textit{D. Blackwell} [Ann. Math. Stat. 36, 226-235 (1965; Zbl 0133.42805)] where the finiteness of the state space was not assumed. (ii) Theorem 6.1, which states that in a communicating MDP the supremum of the average reward per unit time over all policies, is equal the supremum over randomized stationary policies is a particular case of Corollary 4.5 (C) in \textit{E. Feinberg} and \textit{H. Park} [Stochastic Processes Appl. 49, 159-177 (1994; Zbl 0787.60085)] where it was proved for a smaller class of deterministic policies and without additional Assumption 6.1.
- On Stationary Strategies in Borel Dynamic Programming
- A Weighted Markov Decision Process
- Linear programming formulation of MDPs in countable state space: The multichain case
- scientific article; zbMATH DE number 3854831
- Solution to the optimality equation in a class of Markov decision chains with the average cost criterion
- scientific article; zbMATH DE number 700091
- Markov Decision Processes with Sample Path Constraints: The Communicating Case
- Average reward Markov decision programming with a countable action set
- Stationary Policies in Dynamic Programming Models Under Compactness Assumptions
- Semi-infinite discounted Markov decision processes: Policy improvement and singular perturbations
- Empty container management in a port with long-run average criterion
- Special subclass of generalized semi-Markov decision processes with discrete time
- On undiscounted semi-Markov decision processes with absorbing states
- On the reduction of total-cost and average-cost MDPs to discounted mdps
- Semi-Infinite Weighted Markov Decision Processes
- A Weighted Markov Decision Process
- scientific article; zbMATH DE number 1786114 (Why is no real title available?)
- Sufficiency of deterministic policies for atomless discounted and uniformly absorbing MDPs with multiple criteria
This page was built for publication: Semi-infinite Markov decision processes
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1974026)
