Impulsive control for continuous-time Markov decision processes: a linear programming approach

DOI10.1007/s00245-015-9310-8zbMath1347.49031OpenAlexW1170120830MaRDI QIDQ315772

François Dufour, Aleksey B. Piunovskiy

Publication date: 23 September 2016

Published in: Applied Mathematics and Optimization (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1007/s00245-015-9310-8

zbMATH Keywords

optimizationlinear programmingimpulsive controlcontinuous-time Markov decision processdiscounted cost

Mathematics Subject Classification ID

Continuous-time Markov processes on general state spaces (60J25) Linear programming (90C05) Optimal stochastic control (93E20) Markov and semi-Markov decision processes (90C40) Optimality conditions for problems involving randomness (49K45) Impulsive optimal control problems (49N25) Existence of optimal solutions to problems involving randomness (49J55)

Related Items (8)

Simultaneous impulse and continuous control of a Markov chain in continuous time ⋮ On the computation of Whittle's index for Markovian restless bandits ⋮ Gradual-Impulsive Control for Continuous-Time Markov Decision Processes with Total Undiscounted Costs and Constraints: Linear Programming Approach via a Reduction Method ⋮ Asymptotics of impulse control problem with multiplicative reward ⋮ Stability analysis of time-delay discrete systems with logic impulses ⋮ Long-Run Risk-Sensitive Impulse Control ⋮ Design and analysis of two discrete-time ZD algorithms for time-varying nonlinear minimization ⋮ On Reducing a Constrained Gradual-Impulsive Control Problem for a Jump Markov Model to a Model with Gradual Control Only

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• A linear programming formulation for constrained discounted continuous control for piecewise deterministic Markov processes
• Stochastic optimal control and linear programming approach
• Calcul stochastique et problèmes de martingales
• Multicriteria impulsive control of jump Markov processes
• Time-average control of martingale problems: A linear programming formulation
• A survey of recent results on continuous-time Markov decision processes (with comments and rejoinder)
• Continuous-time Markov decision processes. Theory and applications
• On the solution of general impulse control problems using superharmonic functions
• Occupation measures for controlled Markov processes: Characterization and optimality
• Selected Topics on Continuous-Time Controlled Markov Chains and Markov Games
• Ergodic Control of Diffusion Processes
• Discounted Continuous-Time Markov Decision Processes with Unbounded Rates: The Convex Analytic Approach
• The Expected Total Cost Criterion for Markov Decision Processes under Constraints: A Convex Analytic Approach
• Discounted Continuous-Time Markov Decision Processes with Constraints: Unbounded Transition and Loss Rates
• Linear programming approach to the optimal stopping of singular stochastic processes
• Verification Theorems for Markov Decision Processes with Controlled Deterministic Drift and Gradual and Impulsive Controls
• Average optimal policies in Markov decision drift processes with applications to a queueing and a replacement model
• Markov Decision Drift Processes; Conditions for Optimality Obtained by Discretization
• Bellman inequalities in markov decision deterministic drift processes
• Continuous time markov decision processes with interventions
• Numerical Methods for Stochastic Singular Control Problems
• Multivariate point processes: predictable projection, Radon-Nikodym derivatives, representation of martingales
• Existence of Markov Controls and Characterization of Optimal Markov Controls
• Average Optimality in Markov Control Processes via Discounted-Cost Problems and Linear Programming
• Impulsive Control for Continuous-Time Markov Decision Processes
• Ergodic Control of Continuous-Time Markov Chains with Pathwise Constraints

This page was built for publication: Impulsive control for continuous-time Markov decision processes: a linear programming approach