Constrained total undiscounted continuous-time Markov decision processes

DOI10.3150/15-BEJ793MaRDI QIDQ527461zbMATH OpenFDO

Authors Xianping Guo, Yi Zhang

Publication date 11 May 2017

Published in Bernoulli (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1304.3314

zbMATH Keywords

constrained optimalitycontinuous-time Markov decision processestotal undiscounted criteria

Mathematics Subject Classification ID

Continuous-time Markov processes on general state spaces (60J25) Markov and semi-Markov decision processes (90C40)

Abstract: The present paper considers the constrained optimal control problem with total undiscounted criteria for a continuous-time Markov decision process (CTMDP) in Borel state and action spaces. Under the standard compactness and continuity conditions, we show the existence of an optimal stationary policy out of the class of general nonstationary ones. In the process, we justify the reduction of the CTMDP model to a discrete-time Markov decision process (DTMDP) model based on the studies of the undiscounted occupancy and occupation measures. We allow that the controlled process is not necessarily absorbing, and the transition rates are not necessarily separated from zero, and can be arbitrarily unbounded; these features count for the main technical difficulties in studying undiscounted CTMDP models.

Recommendations

• Total reward criteria for unconstrained/constrained continuous-time Markov decision processes
• Discounted continuous-time constrained Markov decision processes in Polish spaces
• Absorbing continuous-time Markov decision processes with total cost criteria
• Constrained Continuous-Time Markov Control Processes with Discounted Criteria
• Discounted continuous-time Markov decision processes with unbounded rates: the convex analytic approach

Cited in

(9)

• Contraction mappings underlying undiscounted Markov decision problems. II
• First passage risk probability minimization for piecewise deterministic Markov decision processes
• A useful technique for piecewise deterministic Markov decision processes
• Discounted Continuous-Time Markov Decision Processes with Constraints: Unbounded Transition and Loss Rates
• Note on discounted continuous-time Markov decision processes with a lower bounding function
• On reducing a constrained gradual-impulsive control problem for a jump Markov model to a model with gradual control only
• Extreme Occupation Measures in Markov Decision Processes with an Absorbing State
• Total reward criteria for unconstrained/constrained continuous-time Markov decision processes
• The minimal hitting probability of continuous-time controlled Markov systems with countable states

This page was built for publication: Constrained total undiscounted continuous-time Markov decision processes

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q527461)