Average continuous control of piecewise deterministic Markov processes

DOI10.1137/080718541zbMATH Open1213.60124arXiv0809.0477OpenAlexW354839235WikidataQ56560046 ScholiaQ56560046MaRDI QIDQ3083221FDOQ3083221

Authors: Oswaldo Luiz V. Costa, F. Dufour 

Publication date: 21 March 2011

Published in: SIAM Journal on Control and Optimization (Search for Journal in Brave)

Abstract: This paper deals with the long run average continuous control problem of piecewise deterministic Markov processes (PDMP's) taking values in a general Borel space and with compact action space depending on the state variable. The control variable acts on the jump rate and transition measure of the PDMP, and the running and boundary costs are assumed to be positive but not necessarily bounded. Our first main result is to obtain an optimality equation for the long run average cost in terms of a discrete-time optimality equation related to the embedded Markov chain given by the post-jump location of the PDMP. Our second main result guarantees the existence of a feedback measurable selector for the discrete-time optimality equation by establishing a connection between this equation and an integro-differential equation. Our final main result is to obtain some sufficient conditions for the existence of a solution for a discrete-time optimality inequality and an ordinary optimal feedback control for the long run average cost using the so-called vanishing discount approach.

Full work available at URL: https://arxiv.org/abs/0809.0477

Recommendations

• scientific article; zbMATH DE number 6174821
• The vanishing discount approach for the average continuous control of piecewise deterministic Markov processes
• Optimal control of piecewise deterministic Markov processes
• Piecewise deterministic Markov control processes with feedback controls and unbounded costs
• Hamilton-Jacobi-Bellman inequality for the average control of piecewise deterministic Markov processes

zbMATH Keywords

piecewise deterministic Markov processoptimal controlcontinuous-timevanishing discount approachlong run average costintegro-differential optimality equation

Mathematics Subject Classification ID

Continuous-time Markov processes on general state spaces (60J25) Markov and semi-Markov decision processes (90C40) Optimal stochastic control (93E20)

Cited In (28)

• Finite-horizon piecewise deterministic Markov decision processes with unbounded transition rates
• Singular perturbation for the discounted continuous control of piecewise deterministic Markov processes
• Optimal control of a class of piecewise deterministic processes
• The vanishing discount approach for the average continuous control of piecewise deterministic Markov processes
• Piecewise deterministic Markov control processes with feedback controls and unbounded costs
• Hermite approximation of a hyperbolic Fokker-Planck optimality system to control a piecewise-deterministic process
• On the vanishing discount factor approach for Markov decision processes with weakly continuous transition probabilities
• On piecewise linear processes
• On the Poisson Equation for Piecewise-Deterministic Markov Processes
• Approximation methods for piecewise deterministic Markov processes and their costs
• Average control of Markov decision processes with Feller transition probabilities and general action spaces
• Asymptotic control for a class of piecewise deterministic Markov processes associated to temperate viruses
• Two-time scale reinforcement learning and applications to production planning
• The policy iteration algorithm for average continuous control of piecewise deterministic Markov processes
• Optimal control of piecewise deterministic Markov processes
• A linear programming formulation for constrained discounted continuous control for piecewise deterministic Markov processes
• Average H 2 control by randomized algorithms
• Hamilton-Jacobi-Bellman inequality for the average control of piecewise deterministic Markov processes
• Adaptive discounted control for piecewise deterministic Markov processes
• Adaptive average control for piecewise deterministic Markov processes
• Optimal control of partially observable piecewise deterministic Markov processes
• A Fokker-Planck control framework for stochastic systems
• Continuous average control of piecewise deterministic Markov processes
• Risk-sensitive finite-horizon piecewise deterministic Markov decision processes
• Uniform asymptotics in the average continuous control of piecewise deterministic Markov processes: vanishing approach
• Abel-type results for controlled piecewise deterministic Markov processes
• Title not available (Why is that?)
• Optimal liquidation under partial information with price impact

This page was built for publication: Average continuous control of piecewise deterministic Markov processes

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3083221)