Constrained total undiscounted continuous-time Markov decision processes

DOI10.3150/15-BEJ793MaRDI QIDQ527461zbMATH OpenFDO

Authors Xianping Guo, Yi Zhang

Publication date 11 May 2017

Published in Bernoulli (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1304.3314

zbMATH Keywords

continuous-time Markov decision processesconstrained optimalitytotal undiscounted criteria

Mathematics Subject Classification ID

Continuous-time Markov processes on general state spaces (60J25) Markov and semi-Markov decision processes (90C40)

Abstract: The present paper considers the constrained optimal control problem with total undiscounted criteria for a continuous-time Markov decision process (CTMDP) in Borel state and action spaces. Under the standard compactness and continuity conditions, we show the existence of an optimal stationary policy out of the class of general nonstationary ones. In the process, we justify the reduction of the CTMDP model to a discrete-time Markov decision process (DTMDP) model based on the studies of the undiscounted occupancy and occupation measures. We allow that the controlled process is not necessarily absorbing, and the transition rates are not necessarily separated from zero, and can be arbitrarily unbounded; these features count for the main technical difficulties in studying undiscounted CTMDP models.

Recommendations

• Total reward criteria for unconstrained/constrained continuous-time Markov decision processes
• Discounted continuous-time constrained Markov decision processes in Polish spaces
• Absorbing continuous-time Markov decision processes with total cost criteria
• Constrained Continuous-Time Markov Control Processes with Discounted Criteria
• Discounted continuous-time Markov decision processes with unbounded rates: the convex analytic approach

Cited in

(8)

• Total reward criteria for unconstrained/constrained continuous-time Markov decision processes
• Discounted Continuous-Time Markov Decision Processes with Constraints: Unbounded Transition and Loss Rates
• A useful technique for piecewise deterministic Markov decision processes
• On reducing a constrained gradual-impulsive control problem for a jump Markov model to a model with gradual control only
• First passage risk probability minimization for piecewise deterministic Markov decision processes
• Extreme Occupation Measures in Markov Decision Processes with an Absorbing State
• Note on discounted continuous-time Markov decision processes with a lower bounding function
• Contraction mappings underlying undiscounted Markov decision problems. II

This page was built for publication: Constrained total undiscounted continuous-time Markov decision processes

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q527461)