Impulsive Control for Continuous-Time Markov Decision Processes

DOI10.1239/AAP/1427814583MaRDI QIDQ5246173zbMATH OpenOpenAlexFDO

Authors F. Dufour, A. B. Piunovskiy

Publication date 17 April 2015

Published in Advances in Applied Probability (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1402.6106, https://projecteuclid.org/euclid.aap/1427814583

zbMATH Keywords

continuous controlcontinuous-time Markov decision processdiscounted costimpulsive control

Mathematics Subject Classification ID

Continuous-time Markov processes on general state spaces (60J25) Markov and semi-Markov decision processes (90C40)

Abstract: The objective of this work is to study continuous-time Markov decision processes on a general Borel state space with both impulsive and continuous controls for the infinite-time horizon discounted cost. The continuous-time controlled process is shown to be non explosive under appropriate hypotheses. The so-called Bellman equation associated to this control problem is studied. Sufficient conditions ensuring the existence and the uniqueness of a bounded measurable solution to this optimality equation are provided. Moreover, it is shown that the value function of the optimization problem under consideration satisfies this optimality equation. Sufficient conditions are also presented to ensure on one hand the existence of an optimal control strategy and on the other hand the existence of an $varepsilon$-optimal control strategy. The decomposition of the state space in two disjoint subsets is exhibited where roughly speaking, one should apply a gradual action or an impulsive action correspondingly to get an optimal or $varepsilon$-optimal strategy. An interesting consequence of our previous results is as follows: the set of strategies that allow interventions at time $t=0$ and only immediately after natural jumps is a sufficient set for the control problem under consideration.

Recommendations

• Impulsive control for continuous-time Markov decision processes: a linear programming approach
• scientific article; zbMATH DE number 3956845
• Impulse control of piecewise deterministic Markov processes
• Impulse and continuous control of piecewise deterministic Markov processes
• Optimal impulsive control of piecewise deterministic Markov processes
• Simultaneous impulse and continuous control of a Markov chain in continuous time
• Optimal strategies for impulse control of piecewise deterministic Markov processes
• Stochastic impulse control of non-Markovian processes

Cites work

• A survey of recent results on continuous-time Markov decision processes (with comments and rejoinder)
• Average optimal policies in Markov decision drift processes with applications to a queueing and a replacement model
• Bellman inequalities in markov decision deterministic drift processes
• Calcul stochastique et problèmes de martingales
• Continuous time markov decision processes with interventions
• Continuous-time Markov decision processes. Theory and applications
• Discretization and Weak Convergence in Markov Decision Drift Processes
• scientific article; zbMATH DE number 3126094 (Why is no real title available?)
• scientific article; zbMATH DE number 3148886 (Why is no real title available?)
• scientific article; zbMATH DE number 3823489 (Why is no real title available?)
• scientific article; zbMATH DE number 1325008 (Why is no real title available?)
• scientific article; zbMATH DE number 802908 (Why is no real title available?)
• scientific article; zbMATH DE number 3301061 (Why is no real title available?)
• Markov Decision Drift Processes; Conditions for Optimality Obtained by Discretization
• Multivariate point processes: predictable projection, Radon-Nikodym derivatives, representation of martingales
• Selected topics on continuous-time controlled Markov chains and Markov games
• Stochastic optimal control. The discrete time case
• Verification Theorems for Markov Decision Processes with Controlled Deterministic Drift and Gradual and Impulsive Controls

Cited in

(18)

• Linear programming approach to optimal impulse control problems with functional constraints
• On the dynamic allocation of assets subject to failure
• On the computation of Whittle's index for Markovian restless bandits
• Impulsive control for continuous-time Markov decision processes: a linear programming approach
• Verification Theorems for Markov Decision Processes with Controlled Deterministic Drift and Gradual and Impulsive Controls
• scientific article; zbMATH DE number 3907509 (Why is no real title available?)
• scientific article; zbMATH DE number 3956845 (Why is no real title available?)
• Realizable strategies in continuous-time Markov decision processes
• On gradual-impulse control of continuous-time Markov decision processes with exponential utility
• On the equivalence of the integral and differential Bellman equations in impulse control problems
• The existence of optimal control for continuous-time Markov decision processes in random environments
• Gradual-Impulsive Control for Continuous-Time Markov Decision Processes with Total Undiscounted Costs and Constraints: Linear Programming Approach via a Reduction Method
• On reducing a constrained gradual-impulsive control problem for a jump Markov model to a model with gradual control only
• Optimal Impulse Control of Dynamical Systems
• Optimal strategies in a production inventory control model
• On the Modeling of Impulse Control with Random Effects for Continuous Markov Processes
• Multicriteria impulsive control of jump Markov processes
• Simultaneous impulse and continuous control of a Markov chain in continuous time

This page was built for publication: Impulsive Control for Continuous-Time Markov Decision Processes

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5246173)