Constrained total undiscounted continuous-time Markov decision processes

DOI10.3150/15-BEJ793zbMATH Open1456.90173arXiv1304.3314MaRDI QIDQ527461FDOQ527461

Authors: Xianping Guo, Yi Zhang 

Publication date: 11 May 2017

Published in: Bernoulli (Search for Journal in Brave)

Abstract: The present paper considers the constrained optimal control problem with total undiscounted criteria for a continuous-time Markov decision process (CTMDP) in Borel state and action spaces. Under the standard compactness and continuity conditions, we show the existence of an optimal stationary policy out of the class of general nonstationary ones. In the process, we justify the reduction of the CTMDP model to a discrete-time Markov decision process (DTMDP) model based on the studies of the undiscounted occupancy and occupation measures. We allow that the controlled process is not necessarily absorbing, and the transition rates are not necessarily separated from zero, and can be arbitrarily unbounded; these features count for the main technical difficulties in studying undiscounted CTMDP models.

Full work available at URL: https://arxiv.org/abs/1304.3314

Recommendations

• Total reward criteria for unconstrained/constrained continuous-time Markov decision processes
• Discounted continuous-time constrained Markov decision processes in Polish spaces
• Absorbing continuous-time Markov decision processes with total cost criteria
• Constrained Continuous-Time Markov Control Processes with Discounted Criteria
• Discounted continuous-time Markov decision processes with unbounded rates: the convex analytic approach

zbMATH Keywords

continuous-time Markov decision processesconstrained optimalitytotal undiscounted criteria

Mathematics Subject Classification ID

Continuous-time Markov processes on general state spaces (60J25) Markov and semi-Markov decision processes (90C40)

Cited In (8)

• Contraction mappings underlying undiscounted Markov decision problems. II
• On reducing a constrained gradual-impulsive control problem for a jump Markov model to a model with gradual control only
• Extreme Occupation Measures in Markov Decision Processes with an Absorbing State
• First passage risk probability minimization for piecewise deterministic Markov decision processes
• Note on discounted continuous-time Markov decision processes with a lower bounding function
• A useful technique for piecewise deterministic Markov decision processes
• Total reward criteria for unconstrained/constrained continuous-time Markov decision processes
• Discounted Continuous-Time Markov Decision Processes with Constraints: Unbounded Transition and Loss Rates

This page was built for publication: Constrained total undiscounted continuous-time Markov decision processes

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q527461)