Optimal control of piecewise deterministic Markov processes: a BSDE representation of the value function

DOI10.1051/COCV/2017006MaRDI QIDQ3177920zbMATH OpenOpenAlexFDO

Authors Elena Bandini

Publication date 2 August 2018

Published in ESAIM: Control, Optimisation and Calculus of Variations (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1512.01659

zbMATH Keywords

backward stochastic differential equationsoptimal control problemsdiscounted costpiecewise deterministic Markov processesrandomization of controls

Mathematics Subject Classification ID

Continuous-time Markov processes on general state spaces (60J25) Stochastic ordinary differential equations (aspects of stochastic analysis) (60H10) Viscosity solutions to Hamilton-Jacobi equations in optimal control and differential games (49L25) Control/observation systems governed by ordinary differential equations (93C15) Optimal stochastic control (93E20) Schrödinger and Feynman-Kac semigroups (47D08)

Abstract: We consider an infinite horizon discounted optimal control problem for piecewise deterministic Markov processes, where a piecewise open-loop control acts continuously on the jump dynamics and on the deterministic flow. For this class of control problems, the value function can in general be characterized as the unique viscosity solution to the corresponding Hamilton-Jacobi-Bellman equation. We prove that the value function can be represented by means of a backward stochastic differential equation (BSDE) on infinite horizon, driven by a random measure and with a sign constraint on its martingale part, for which we give existence and uniqueness results. This probabilistic representation is known as nonlinear Feynman-Kac formula. Finally we show that the constrained BSDE is related to an auxiliary dominated control problem, whose value function coincides with the value function of the original non-dominated control problem.

Recommendations

• Constrained BSDEs representation of the value function in optimal control of pure jump Markov processes
• Constrained BSDEs driven by a non-quasi-left-continuous random measure and optimal control of PDMPs on bounded domains
• Optimal control of infinite-dimensional piecewise deterministic Markov processes: a BSDE approach. Application to the control of an excitable cell membrane
• Backward stochastic differential equations and optimal control of marked point processes
• Backward SDEs and infinite horizon stochastic optimal control

Cites work

• scientific article; zbMATH DE number 425394 (Why is no real title available?)
• scientific article; zbMATH DE number 3664138 (Why is no real title available?)
• scientific article; zbMATH DE number 3793150 (Why is no real title available?)
• scientific article; zbMATH DE number 17494 (Why is no real title available?)
• scientific article; zbMATH DE number 1245556 (Why is no real title available?)
• scientific article; zbMATH DE number 1254171 (Why is no real title available?)
• scientific article; zbMATH DE number 481040 (Why is no real title available?)
• scientific article; zbMATH DE number 1834045 (Why is no real title available?)
• A dynamic programming algorithm for the optimal control of piecewise deterministic Markov processes
• A numerical algorithm for fully nonlinear HJB equations: an approach by control randomization
• BSDE representations for optimal switching problems with controlled volatility
• BSDEs with diffusion constraint and viscous Hamilton-Jacobi equations with unbounded data
• Backward SDE representation for stochastic control problems with nondominated controlled intensity
• Backward SDEs with constrained jumps and quasi-variational inequalities
• Backward Stochastic Differential Equations in Finance
• Backward stochastic differential equation driven by a marked point process: an elementary approach with an application to optimal control
• Backward stochastic differential equation with random measures
• Backward stochastic differential equations and integral-partial differential equations
• Backward stochastic differential equations and optimal control of marked point processes
• Backward stochastic differential equations associated to jump Markov processes and applications
• Bounded solutions to backward SDEs with jumps for utility optimization and indifference hedging
• Calcul stochastique et problèmes de martingales
• Constrained BSDEs representation of the value function in optimal control of pure jump Markov processes
• Continuous average control of piecewise deterministic Markov processes
• Discrete time approximation of fully nonlinear HJB equations via BSDEs with nonpositive jumps
• Existence and uniqueness for backward stochastic differential equations driven by a random measure, possibly non quasi-left continuous
• Existence, uniqueness and comparisons for BSDEs in general spaces
• Feynman-Kac representation for Hamilton-Jacobi-Bellman IPDE
• Functional analysis, Sobolev spaces and partial differential equations
• Long time asymptotics for fully nonlinear Bellman equations: a backward SDE approach
• Multivariate point processes: predictable projection, Radon-Nikodym derivatives, representation of martingales
• Necessary and sufficient optimality conditions for control of piecewise deterministic markov processes
• Optimal Control with State-Space Constraint. II
• Optimal control of piecewise deterministic markov process
• Optimal control of semi-Markov processes with a backward stochastic differential equations approach
• Point processes and queues. Martingale dynamics
• Probabilistic representation and approximation for coupled systems of variational inequalities
• Probability essentials.
• Randomized and backward SDE representation for optimal control of non-Markovian SDEs
• Reflected BSDEs with nonpositive jumps, and controller-and-stopper games
• Special weak Dirichlet processes and BSDEs driven by a random measure
• Weak Dirichlet processes with jumps

Cited in

(16)

• Constrained BSDEs driven by a non-quasi-left-continuous random measure and optimal control of PDMPs on bounded domains
• Optimal control of infinite-dimensional piecewise deterministic Markov processes: a BSDE approach. Application to the control of an excitable cell membrane
• Optimal control of continuous-time Markov chains with noise-free observation
• Representation of non-Markovian optimal stopping problems by constrained BSDEs with a single jump
• Backward SDEs and infinite horizon stochastic optimal control
• The identification problem for BSDEs driven by possibly non-quasi-left-continuous random measures
• A Tikhonov theorem for McKean-Vlasov two-scale systems and a new application to mean field optimal control problems
• Optimal switching problems with an infinite set of modes: an approach by randomization and constrained backward SDEs
• Optimal control of semi-Markov processes with a backward stochastic differential equations approach
• Special weak Dirichlet processes and BSDEs driven by a random measure
• Optimal stopping of marked point processes and reflected backward stochastic differential equations
• BSDE representation and randomized dynamic programming principle for stochastic control problems of infinite-dimensional jump-diffusions
• Backward SDEs for optimal control of partially observed path-dependent stochastic systems: A control randomization approach
• Stochastic filtering and optimal control of pure jump Markov processes with noise-free partial observation
• Weak Dirichlet processes and generalized martingale problems
• Constrained BSDEs representation of the value function in optimal control of pure jump Markov processes

This page was built for publication: Optimal control of piecewise deterministic Markov processes: a BSDE representation of the value function

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3177920)