Discounted continuous-time constrained Markov decision processes in Polish spaces
From MaRDI portal
Publication:655591
DOI10.1214/10-AAP749zbMATH Open1258.90104arXiv1201.0089MaRDI QIDQ655591FDOQ655591
Publication date: 4 January 2012
Published in: The Annals of Applied Probability (Search for Journal in Brave)
Abstract: This paper is devoted to studying constrained continuous-time Markov decision processes (MDPs) in the class of randomized policies depending on state histories. The transition rates may be unbounded, the reward and costs are admitted to be unbounded from above and from below, and the state and action spaces are Polish spaces. The optimality criterion to be maximized is the expected discounted rewards, and the constraints can be imposed on the expected discounted costs. First, we give conditions for the nonexplosion of underlying processes and the finiteness of the expected discounted rewards/costs. Second, using a technique of occupation measures, we prove that the constrained optimality of continuous-time MDPs can be transformed to an equivalent (optimality) problem over a class of probability measures. Based on the equivalent problem and a so-called -weak convergence of probability measures developed in this paper, we show the existence of a constrained optimal policy. Third, by providing a linear programming formulation of the equivalent problem, we show the solvability of constrained optimal policies. Finally, we use two computable examples to illustrate our main results.
Full work available at URL: https://arxiv.org/abs/1201.0089
Recommendations
- Continuous-Time Markov Decision Processes with Discounted Rewards: The Case of Polish Spaces
- Constrained optimality for finite horizon semi-Markov decision processes in Polish spaces
- Constrained discounted Markov decision processes with Borel state spaces
- Constrained Continuous-Time Markov Control Processes with Discounted Criteria
- Average optimality for continuous-time Markov decision processes in Polish spaces
- Constrained continuous-time Markov decision processes on the finite horizon
- Discounted Continuous-Time Markov Decision Processes with Constraints: Unbounded Transition and Loss Rates
- Continuous time Markov decision processes with discounted moment criterion
- Constrained optimality problem of Markov decision processes with Borel spaces and varying discount factors
- Constrained semi-Markov decision processes with ratio and time expected average criteria in Polish spaces
occupation measureunbounded transition ratescontinuous-time Markov decision processlinear programming formulationconstrained optimal policy
Continuous-time Markov processes on discrete state spaces (60J27) Markov and semi-Markov decision processes (90C40)
Cites Work
- Title not available (Why is that?)
- Multivariate point processes: predictable projection, Radon-Nikodym derivatives, representation of martingales
- Title not available (Why is that?)
- Bias Optimality in Controlled Queueing Systems
- Continuous-time Markov decision processes. Theory and applications
- Stochastic optimal control. The discrete time case
- Title not available (Why is that?)
- Lectures on Choquet's theorem
- A survey of recent results on continuous-time Markov decision processes (with comments and rejoinder)
- Title not available (Why is that?)
- Title not available (Why is that?)
- Constrained Optimization for Average Cost Continuous-Time Markov Decision Processes
- Title not available (Why is that?)
- Constrained Markov control processes in Borel spaces: the discounted case
- Constrained discounted Markov decision processes and Hamiltonian cycles.
- Constrained Discounted Markov Decision Chains
- Constrained admission control to a queueing system
- Constrained Average Cost Markov Control Processes in Borel Spaces
- Constrained dynamic programming with two discount factors: applications and an algorithm
- Constrained markov decision processes with compact state and action spaces: the average case
- Constrained Continuous-Time Markov Control Processes with Discounted Criteria
- Robustness of Policies in Constrained Markov Decision Processes
- Semi-Markov and Jump Markov Controlled Models: Average Cost Criterion
- Title not available (Why is that?)
- Controlled Markov Models with Countable State Space and Continuous Time
- Title not available (Why is that?)
- Ergodic Control of Continuous-Time Markov Chains with Pathwise Constraints
- Computable exponential convergence rates for stochastically ordered Markov processes
- Constrained Discounted Dynamic Programming
- Constrained Markov Decision Models with Weighted Discounted Rewards
- Average optimality for continuous-time Markov decision processes in Polish spaces
- Constrained continuous-time Markov decision processes with average criteria
- Title not available (Why is that?)
- Non-randomized policies for constrained Markov decision processes
- Discounted Markov decision processes with utility constraints
- Continuous-Time Markov Decision Processes with Discounted Rewards: The Case of Polish Spaces
- Continuously Discounted Markov Decision Model with Countable State and Action Space
- Управляемая скачкообразная модель с дисконтированием при наличии ограничений
- Continuous-time controlled Markov chains.
- Markov Decision Problems and State-Action Frequencies
- A probabilistic analysis of bias optimality in unichain Markov decision processes
Cited In (29)
- Absorbing continuous-time Markov decision processes with total cost criteria
- Finite-horizon piecewise deterministic Markov decision processes with unbounded transition rates
- Risk-sensitive average continuous-time Markov decision processes with unbounded transition and cost rates
- Constrained expected average stochastic games for continuous-time jump processes
- Continuous-time Markov decision processes with state-dependent discount factors
- A mean-variance optimization problem for discounted Markov decision processes
- Constrained Continuous-Time Markov Control Processes with Discounted Criteria
- Zero-sum infinite-horizon discounted piecewise deterministic Markov games
- Risk-Sensitive Discounted Continuous-Time Markov Decision Processes with Unbounded Rates
- Constrained semi-Markov decision processes with ratio and time expected average criteria in Polish spaces
- Continuous-time zero-sum games for Markov chains with risk-sensitive finite-horizon cost criterion
- Nonzero-sum risk-sensitive continuous-time stochastic games with ergodic costs
- Bias and Overtaking Optimality for Continuous-Time Jump Markov Decision Processes in Polish Spaces
- Title not available (Why is that?)
- Continuous Time Discounted Jump Markov Decision Processes: A Discrete-Event Approach
- Discounted continuous-time Markov decision processes with unbounded rates and randomized history-dependent policies: the dynamic programming approach
- Continuous-time zero-sum games for markov decision processes with discounted risk-sensitive cost criterion on a general state space
- Risk-sensitive continuous-time Markov decision processes with unbounded rates and Borel spaces
- A linear programming formulation for constrained discounted continuous control for piecewise deterministic Markov processes
- First passage risk probability minimization for piecewise deterministic Markov decision processes
- Continuous-time zero-sum games for Markov decision processes with discounted risk-sensitive cost criterion
- Continuous-time Markov decision processes under the risk-sensitive first passage discounted cost criterion
- Kolmogorov's Equations for Jump Markov Processes and Their Applications to Control Problems
- Discounted stochastic games for continuous-time jump processes with an uncountable state space
- A dynamic analytic method for risk-aware controlled martingale problems
- Constrained continuous-time Markov decision processes on the finite horizon
- Sufficiency of Markov Policies for Continuous-Time Jump Markov Decision Processes
- Asymptotic optimality and rates of convergence of quantized stationary policies in continuous-time Markov decision processes
- On the First Passage $g$-Mean-Variance Optimality for Discounted Continuous-Time Markov Decision Processes
This page was built for publication: Discounted continuous-time constrained Markov decision processes in Polish spaces
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q655591)
