Constrained BSDEs representation of the value function in optimal control of pure jump Markov processes

DOI10.1016/J.SPA.2016.08.005zbMATH Open1362.93163arXiv1501.04362OpenAlexW2267376484MaRDI QIDQ529424FDOQ529424

Authors: Elena Bandini, Marco Fuhrman 

Publication date: 18 May 2017

Published in: Stochastic Processes and their Applications (Search for Journal in Brave)

Abstract: We consider a classical finite horizon optimal control problem for continuous-time pure jump Markov processes described by means of a rate transition measure depending on a control parameter and controlled by a feedback law. For this class of problems the value function can often be described as the unique solution to the corresponding Hamilton-Jacobi-Bellman equation. We prove a probabilistic representation for the value function, known as nonlinear Feynman-Kac formula. It relates the value function with a backward stochastic differential equation (BSDE) driven by a random measure and with a sign constraint on its martingale part. We also prove existence and uniqueness results for this class of constrained BSDEs. The connection of the control problem with the constrained BSDE uses a control randomization method recently developed in the works of I. Kharroubi and H. Pham and their co-authors. This approach also allows to prove that the value function of the original non-dominated control problem coincides with the value function of an auxiliary dominated control problem, expressed in terms of equivalent changes of probability measures.

Full work available at URL: https://arxiv.org/abs/1501.04362

Recommendations

• Optimal control of piecewise deterministic Markov processes: a BSDE representation of the value function
• Constrained BSDEs driven by a non-quasi-left-continuous random measure and optimal control of PDMPs on bounded domains
• Backward stochastic differential equations associated to jump Markov processes and applications
• Representation of non-Markovian optimal stopping problems by constrained BSDEs with a single jump
• Backward stochastic differential equations and optimal control of marked point processes

zbMATH Keywords

randomizationbackward stochastic differential equationsoptimal control problemsmarked point processespure jump Markov processes

Mathematics Subject Classification ID

Point processes (e.g., Poisson, Cox, Hawkes processes) (60G55) Continuous-time Markov processes on general state spaces (60J25) Stochastic ordinary differential equations (aspects of stochastic analysis) (60H10) Optimal stochastic control (93E20)

Cites Work

• Multivariate point processes: predictable projection, Radon-Nikodym derivatives, representation of martingales
• Second-order backward stochastic differential equations and fully nonlinear parabolic PDEs
• Backward Stochastic Differential Equations in Finance
• Wellposedness of second order backward SDEs
• \(G\)-expectation, \(G\)-Brownian motion and related stochastic calculus of Itô type
• Continuous-time Markov decision processes. Theory and applications
• Stochastic optimal control. The discrete time case
• Title not available (Why is that?)
• Applied functional analysis. Functional analysis, Sobolev spaces and elliptic differential equations
• Bounded solutions to backward SDEs with jumps for utility optimization and indifference hedging
• Point processes and queues. Martingale dynamics
• Feynman-Kac representation for Hamilton-Jacobi-Bellman IPDE
• Long time asymptotics for fully nonlinear Bellman equations: a backward SDE approach
• Reflected backward stochastic differential equation with jumps and RCLL obstacle
• A numerical algorithm for fully nonlinear HJB equations: an approach by control randomization
• Randomized and backward SDE representation for optimal control of non-Markovian SDEs
• Discrete time approximation of fully nonlinear HJB equations via BSDEs with nonpositive jumps
• Backward stochastic differential equations and optimal control of marked point processes
• Backward SDE representation for stochastic control problems with nondominated controlled intensity
• Backward SDEs with constrained jumps and quasi-variational inequalities
• Backward stochastic differential equation with random measures
• Backward stochastic differential equations associated to jump Markov processes and applications
• No Arbitrage and General Semimartingales
• Optimal control of semi-Markov processes with a backward stochastic differential equations approach
• Optimal high-frequency trading with limit and market orders
• Title not available (Why is that?)
• Optimal Control of Jump Processes
• A stochastic target formulation for optimal switching problems in finite horizon
• Probabilistic representation and approximation for coupled systems of variational inequalities
• Modeling high-frequency financial data by pure jump processes
• Adding constraints to BSDEs with jumps: an alternative to multidimensional reflections
• Controlled jump processes
• BSDE representations for optimal switching problems with controlled volatility

Cited In (19)

• BSDEs with Singular Terminal Condition and a Control Problem with Constraints
• Constrained BSDEs driven by a non-quasi-left-continuous random measure and optimal control of PDMPs on bounded domains
• An approximate Nash equilibrium for pure jump Markov games of mean-field-type on continuous state space
• Optimal control of infinite-dimensional piecewise deterministic Markov processes: a BSDE approach. Application to the control of an excitable cell membrane
• Backward SDEs and infinite horizon stochastic optimal control
• Representation of non-Markovian optimal stopping problems by constrained BSDEs with a single jump
• The identification problem for BSDEs driven by possibly non-quasi-left-continuous random measures
• A Tikhonov theorem for McKean-Vlasov two-scale systems and a new application to mean field optimal control problems
• Optimal switching problems with an infinite set of modes: an approach by randomization and constrained backward SDEs
• Optimal control of semi-Markov processes with a backward stochastic differential equations approach
• Special weak Dirichlet processes and BSDEs driven by a random measure
• Randomized filtering and Bellman equation in Wasserstein space for partial observation control problem
• BSDE representation and randomized dynamic programming principle for stochastic control problems of infinite-dimensional jump-diffusions
• Backward SDEs for optimal control of partially observed path-dependent stochastic systems: A control randomization approach
• Weak Dirichlet processes with jumps
• Stochastic filtering and optimal control of pure jump Markov processes with noise-free partial observation
• Title not available (Why is that?)
• Optimal control of piecewise deterministic Markov processes: a BSDE representation of the value function
• BSDE representations for optimal switching problems with controlled volatility

This page was built for publication: Constrained BSDEs representation of the value function in optimal control of pure jump Markov processes

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q529424)