Discounted continuous-time Markov decision processes with unbounded rates and randomized history-dependent policies: the dynamic programming approach

DOI10.1007/S10288-013-0236-1zbMATH Open1297.90174arXiv1103.0134OpenAlexW2016588739MaRDI QIDQ457293FDOQ457293

Authors: Yi Zhang, A. B. Piunovskiy 

Publication date: 26 September 2014

Published in: 4OR (Search for Journal in Brave)

Abstract: This paper deals with unconstrained discounted continuous-time Markov decision processes in Borel state and action spaces. Under some conditions imposed on the primitives, allowing unbounded transition rates and unbounded (from both above and below) cost rates, we show the regularity of the controlled process, which ensures the underlying models to be well defined. Then we develop the dynamic programming approach by showing that the Bellman equation is satisfied (by the optimal value). Finally, under some compactness-continuity conditions, we obtain the existence of a deterministic stationary optimal policy out of the class of randomized history-dependent policies.

Full work available at URL: https://arxiv.org/abs/1103.0134

Recommendations

• Discounted continuous-time Markov decision processes with unbounded rates: the convex analytic approach
• Nonstationary continuous time Markov decision processes with discounted criterion
• Continuous-time Markov decision processes with state-dependent discount factors
• Continuous-Time Markov Decision Processes with Unbounded Transition and Discounted-Reward Rates
• Continuous-Time Markov Decision Processes with Discounted Rewards: The Case of Polish Spaces

zbMATH Keywords

dynamic programmingBellman equationcontinuous-time Markov decision processdynkin's formula

Mathematics Subject Classification ID

Continuous-time Markov processes on general state spaces (60J25) Markov and semi-Markov decision processes (90C40)

Cites Work

• Title not available (Why is that?)
• Multivariate point processes: predictable projection, Radon-Nikodym derivatives, representation of martingales
• Title not available (Why is that?)
• Continuous-time Markov decision processes. Theory and applications
• Stochastic optimal control. The discrete time case
• Selected topics on continuous-time controlled Markov chains and Markov games
• A survey of recent results on continuous-time Markov decision processes (with comments and rejoinder)
• The transformation method for continuous-time Markov decision processes
• Denumerable-state continuous-time Markov decision processes with unbounded transition and reward rates under the discounted criterion
• Discounted Continuous-Time Markov Decision Processes with Constraints: Unbounded Transition and Loss Rates
• Reduction of Discounted Continuous-Time MDPs with Unbounded Jump and Reward Rates to Discrete-Time Total-Reward MDPs
• Continuous Time Discounted Jump Markov Decision Processes: A Discrete-Event Approach
• Discounted Continuous-Time Markov Decision Processes with Unbounded Rates: The Convex Analytic Approach
• Semi-Markov and Jump Markov Controlled Models: Average Cost Criterion
• Discounted continuous-time Markov decision processes with unbounded rates and randomized history-dependent policies: the dynamic programming approach
• Title not available (Why is that?)
• Continuous-Time Markov Decision Processes with Unbounded Transition and Discounted-Reward Rates
• Absorbing continuous-time Markov decision processes with total cost criteria
• Linear programming and constrained average optimality for general continuous-time Markov decision processes in history-dependent policies
• Accuracy of fluid approximations to controlled birth-and-death processes: absorbing case
• Continuous-Time Markov Decision Processes with Discounted Rewards: The Case of Polish Spaces
• Continuously Discounted Markov Decision Model with Countable State and Action Space
• Discounted continuous-time constrained Markov decision processes in Polish spaces

Cited In (17)

• Finite-horizon piecewise deterministic Markov decision processes with unbounded transition rates
• Time-varying Markov decision processes with state-action-dependent discount factors and unbounded costs
• On the link between infinite horizon control and quasi-stationary distributions
• Finite horizon continuous-time Markov decision processes with mean and variance criteria
• Computable approximations for continuous-time Markov decision processes on Borel spaces based on empirical measures
• Title not available (Why is that?)
• Discounted continuous-time Markov decision processes with unbounded rates and randomized history-dependent policies: the dynamic programming approach
• First passage risk probability minimization for piecewise deterministic Markov decision processes
• Optimal strategies in a production inventory control model
• Finite horizon risk-sensitive continuous-time Markov decision processes with unbounded transition and cost rates
• Kolmogorov's Equations for Jump Markov Processes and Their Applications to Control Problems
• On the nonexplosion and explosion for nonhomogeneous Markov pure jump processes
• Discounted stochastic games for continuous-time jump processes with an uncountable state space
• Computable approximations for average Markov decision processes in continuous time
• Sufficiency of Markov Policies for Continuous-Time Jump Markov Decision Processes
• Discounted Continuous-Time Markov Decision Processes with Constraints: Unbounded Transition and Loss Rates
• Countable state Markov decision processes with unbounded jump rates and discounted cost: optimality equation and approximations

This page was built for publication: Discounted continuous-time Markov decision processes with unbounded rates and randomized history-dependent policies: the dynamic programming approach

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q457293)