Optimal control of piecewise deterministic Markov processes: a BSDE representation of the value function

DOI10.1051/COCV/2017006zbMATH Open1396.93131arXiv1512.01659OpenAlexW2963553379MaRDI QIDQ3177920FDOQ3177920

Elena Bandini

Publication date: 2 August 2018

Published in: ESAIM: Control, Optimisation and Calculus of Variations (Search for Journal in Brave)

Abstract: We consider an infinite horizon discounted optimal control problem for piecewise deterministic Markov processes, where a piecewise open-loop control acts continuously on the jump dynamics and on the deterministic flow. For this class of control problems, the value function can in general be characterized as the unique viscosity solution to the corresponding Hamilton-Jacobi-Bellman equation. We prove that the value function can be represented by means of a backward stochastic differential equation (BSDE) on infinite horizon, driven by a random measure and with a sign constraint on its martingale part, for which we give existence and uniqueness results. This probabilistic representation is known as nonlinear Feynman-Kac formula. Finally we show that the constrained BSDE is related to an auxiliary dominated control problem, whose value function coincides with the value function of the original non-dominated control problem.

Full work available at URL: https://arxiv.org/abs/1512.01659

Recommendations

• Constrained BSDEs representation of the value function in optimal control of pure jump Markov processes
• Constrained BSDEs driven by a non-quasi-left-continuous random measure and optimal control of PDMPs on bounded domains
• Optimal control of infinite-dimensional piecewise deterministic Markov processes: a BSDE approach. Application to the control of an excitable cell membrane
• Backward stochastic differential equations and optimal control of marked point processes
• Backward SDEs and infinite horizon stochastic optimal control

zbMATH Keywords

backward stochastic differential equationsoptimal control problemsdiscounted costpiecewise deterministic Markov processesrandomization of controls

Mathematics Subject Classification ID

Continuous-time Markov processes on general state spaces (60J25) Stochastic ordinary differential equations (aspects of stochastic analysis) (60H10) Viscosity solutions to Hamilton-Jacobi equations in optimal control and differential games (49L25) Control/observation systems governed by ordinary differential equations (93C15) Optimal stochastic control (93E20) Schrödinger and Feynman-Kac semigroups (47D08)

Cites Work

• Multivariate point processes: predictable projection, Radon-Nikodym derivatives, representation of martingales
• Title not available (Why is that?)
• Title not available (Why is that?)
• Functional analysis, Sobolev spaces and partial differential equations
• Optimal control of piecewise deterministic markov process
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Backward Stochastic Differential Equations in Finance
• Backward stochastic differential equations and integral-partial differential equations
• Calcul stochastique et problèmes de martingales
• Bounded solutions to backward SDEs with jumps for utility optimization and indifference hedging
• Point processes and queues. Martingale dynamics
• Probability essentials.
• Feynman-Kac representation for Hamilton-Jacobi-Bellman IPDE
• Long time asymptotics for fully nonlinear Bellman equations: a backward SDE approach
• Title not available (Why is that?)
• A numerical algorithm for fully nonlinear HJB equations: an approach by control randomization
• Randomized and backward SDE representation for optimal control of non-Markovian SDEs
• Discrete time approximation of fully nonlinear HJB equations via BSDEs with nonpositive jumps
• Reflected BSDEs with nonpositive jumps, and controller-and-stopper games
• Backward stochastic differential equations and optimal control of marked point processes
• Backward SDE representation for stochastic control problems with nondominated controlled intensity
• Backward SDEs with constrained jumps and quasi-variational inequalities
• Backward stochastic differential equation with random measures
• Backward stochastic differential equations associated to jump Markov processes and applications
• Backward stochastic differential equation driven by a marked point process: an elementary approach with an application to optimal control
• Optimal Control with State-Space Constraint. II
• Optimal control of semi-Markov processes with a backward stochastic differential equations approach
• Continuous Average Control of Piecewise Deterministic Markov Processes
• Existence, uniqueness and comparisons for BSDEs in general spaces
• Probabilistic representation and approximation for coupled systems of variational inequalities
• A dynamic programming algorithm for the optimal control of piecewise deterministic Markov processes
• Title not available (Why is that?)
• Necessary and sufficient optimality conditions for control of piecewise deterministic markov processes
• BSDE representations for optimal switching problems with controlled volatility
• Constrained BSDEs representation of the value function in optimal control of pure jump Markov processes
• Existence and uniqueness for backward stochastic differential equations driven by a random measure, possibly non quasi-left continuous
• Special weak Dirichlet processes and BSDEs driven by a random measure
• Weak Dirichlet processes with jumps
• BSDEs with diffusion constraint and viscous Hamilton-Jacobi equations with unbounded data
• Title not available (Why is that?)

Cited In (12)

• Optimal control of infinite-dimensional piecewise deterministic Markov processes: a BSDE approach. Application to the control of an excitable cell membrane
• Backward SDEs and infinite horizon stochastic optimal control
• The identification problem for BSDEs driven by possibly non-quasi-left-continuous random measures
• A Tikhonov theorem for McKean-Vlasov two-scale systems and a new application to mean field optimal control problems
• Optimal switching problems with an infinite set of modes: an approach by randomization and constrained backward SDEs
• Special weak Dirichlet processes and BSDEs driven by a random measure
• Optimal stopping of marked point processes and reflected backward stochastic differential equations
• Backward SDEs for optimal control of partially observed path-dependent stochastic systems: A control randomization approach
• Stochastic filtering and optimal control of pure jump Markov processes with noise-free partial observation
• Optimal Control of Continuous-Time Markov Chains with Noise-Free Observation
• Constrained BSDEs Driven by a Non-Quasi-Left-Continuous Random Measure and Optimal Control of PDMPs on Bounded Domains
• Weak Dirichlet processes and generalized martingale problems

This page was built for publication: Optimal control of piecewise deterministic Markov processes: a BSDE representation of the value function

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3177920)