Discounted continuous-time Markov decision processes with unbounded rates and randomized history-dependent policies: the dynamic programming approach

DOI10.1007/S10288-013-0236-1zbMath1297.90174arXiv1103.0134OpenAlexW2016588739MaRDI QIDQ457293

Yi Zhang, Aleksey B. Piunovskiy

Publication date: 26 September 2014

Published in: 4OR (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1103.0134

zbMATH Keywords

dynamic programmingBellman equationcontinuous-time Markov decision processdynkin's formula

Mathematics Subject Classification ID

Continuous-time Markov processes on general state spaces (60J25) Markov and semi-Markov decision processes (90C40)

Related Items (14)

Kolmogorov's Equations for Jump Markov Processes and Their Applications to Control Problems ⋮ Finite horizon continuous-time Markov decision processes with mean and variance criteria ⋮ Computable approximations for continuous-time Markov decision processes on Borel spaces based on empirical measures ⋮ Discounted stochastic games for continuous-time jump processes with an uncountable state space ⋮ First passage risk probability minimization for piecewise deterministic Markov decision processes ⋮ Sufficiency of Markov Policies for Continuous-Time Jump Markov Decision Processes ⋮ Finite-horizon piecewise deterministic Markov decision processes with unbounded transition rates ⋮ Optimal strategies in a production inventory control model ⋮ Discounted continuous-time Markov decision processes with unbounded rates and randomized history-dependent policies: the dynamic programming approach ⋮ On the link between infinite horizon control and quasi-stationary distributions ⋮ Countable state Markov decision processes with unbounded jump rates and discounted cost: optimality equation and approximations ⋮ Computable approximations for average Markov decision processes in continuous time ⋮ Finite horizon risk-sensitive continuous-time Markov decision processes with unbounded transition and cost rates ⋮ On the nonexplosion and explosion for nonhomogeneous Markov pure jump processes

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Discounted continuous-time Markov decision processes with unbounded rates and randomized history-dependent policies: the dynamic programming approach
• Accuracy of fluid approximations to controlled birth-and-death processes: absorbing case
• Discounted continuous-time constrained Markov decision processes in Polish spaces
• A survey of recent results on continuous-time Markov decision processes (with comments and rejoinder)
• Continuous-time Markov decision processes. Theory and applications
• Stochastic optimal control. The discrete time case
• The transformation method for continuous-time Markov decision processes
• Absorbing Continuous-Time Markov Decision Processes with Total Cost Criteria
• Linear Programming and Constrained Average Optimality for General Continuous-Time Markov Decision Processes in History-Dependent Policies
• Selected Topics on Continuous-Time Controlled Markov Chains and Markov Games
• Discounted Continuous-Time Markov Decision Processes with Unbounded Rates: The Convex Analytic Approach
• Denumerable-state continuous-time Markov decision processes with unbounded transition and reward rates under the discounted criterion
• Discounted Continuous-Time Markov Decision Processes with Constraints: Unbounded Transition and Loss Rates
• Semi-Markov and Jump Markov Controlled Models: Average Cost Criterion
• Multivariate point processes: predictable projection, Radon-Nikodym derivatives, representation of martingales
• Reduction of Discounted Continuous-Time MDPs with Unbounded Jump and Reward Rates to Discrete-Time Total-Reward MDPs
• Continuous-Time Markov Decision Processes with Discounted Rewards: The Case of Polish Spaces
• Continuous-Time Markov Decision Processes with Unbounded Transition and Discounted-Reward Rates
• Continuously Discounted Markov Decision Model with Countable State and Action Space
• Continuous Time Discounted Jump Markov Decision Processes: A Discrete-Event Approach

This page was built for publication: Discounted continuous-time Markov decision processes with unbounded rates and randomized history-dependent policies: the dynamic programming approach