Optimal control of Markov processes with age-dependent transition rates

DOI10.1007/S00245-012-9171-3MaRDI QIDQ2391935zbMATH OpenOpenAlexFDO

Authors Subhamay Saha, Mrinal K. Ghosh

Publication date 5 August 2013

Published in Applied Mathematics and Optimization (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1409.4028

zbMATH Keywords

age-dependent transition ratesaverage cost casescontrol policyinfinite horizon average costinfinite horizon discounted costsemi-Markov decision processvalue function

Mathematics Subject Classification ID

Optimal stochastic control (93E20)

Abstract: We study optimal control of Markov processes with age-dependent transition rates. The control policy is chosen continuously over time based on the state of the process and its age. We study infinite horizon discounted cost and infinite horizon average cost problems. Our approach is via the construction of an equivalent semi-Markov decision process. We characterize the value function and optimal controls for both discounted and average cost cases.

Recommendations

• scientific article; zbMATH DE number 3941292
• scientific article; zbMATH DE number 1424150
• scientific article; zbMATH DE number 4128758
• Nonstationary continuous-time Markov control processes with discounted costs on infinite horizon
• scientific article; zbMATH DE number 3926784

Cites work

• A survey of recent results on continuous-time Markov decision processes (with comments and rejoinder)
• Arbitrary state semi-Markov decision processes
• Average optimal policies in Markov decision drift processes with applications to a queueing and a replacement model
• Continuous-time controlled Markov chains.
• Continuous-time Markov decision processes. Theory and applications
• Controlled jump processes
• Denumerable state semi-Markov decision processes with unbounded costs, average cost criterion
• Denumerable Undiscounted Semi-Markov Decision Processes with Unbounded Rewards
• Discretization and Weak Convergence in Markov Decision Drift Processes
• scientific article; zbMATH DE number 425394 (Why is no real title available?)
• scientific article; zbMATH DE number 5728694 (Why is no real title available?)
• scientific article; zbMATH DE number 3951715 (Why is no real title available?)
• scientific article; zbMATH DE number 43057 (Why is no real title available?)
• scientific article; zbMATH DE number 3621618 (Why is no real title available?)
• Markov Decision Drift Processes; Conditions for Optimality Obtained by Discretization
• Risk Minimizing Option Pricing in a Semi-Markov Modulated Market

Cited in

(4)

• Using age structure for a multi-stage optimal control model with random switching time
• Infinite horizon optimal control of mean-field delay system with semi-Markov modulated jump-diffusion processes
• scientific article; zbMATH DE number 3941292 (Why is no real title available?)
• An optimal threshold policy in applications of a two-state Markov process

This page was built for publication: Optimal control of Markov processes with age-dependent transition rates

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2391935)