Optimal control of semi-Markov processes with a backward stochastic differential equations approach

DOI10.1007/S00498-016-0181-6zbMATH Open1360.93764DBLPjournals/mcss/BandiniC17arXiv1311.1063OpenAlexW1552220249WikidataQ56527066 ScholiaQ56527066MaRDI QIDQ525049FDOQ525049

Authors: Elena Bandini, F. Confortola 

Publication date: 28 April 2017

Published in: MCSS. Mathematics of Control, Signals, and Systems (Search for Journal in Brave)

Abstract: In the present work we employ, for the first time, backward stochastic differential equations (BSDEs) to study the optimal control of semi-Markov processes on finite horizon, with general state and action spaces. More precisely, we prove that the value function and the optimal control law can be represented by means of the solution of a class of BSDEs driven by a semi-Markov process or, equivalently, by the associated random measure. The peculiarity of the semi-Markov framework, with respect to the pure jump Markov case, consists in the proof of the relation between BSDE and optimal control problem. This is done, as usual, via the Hamilton-Jacobi-Bellman (HJB) equation, which however in the semi-Markov case is characterized by an additional differential term $partia{l}_a$. Taking into account the particular structure of semi-Markov processes we rewrite the HJB equation in a suitable integral form which involves a directional derivative operator $D$ related to $partia{l}_a$. Then, using a formula of Ito type tailor-made for semi-Markov processes and the operator $D$, we are able to prove that the BSDE provides the unique classical solution to the HJB equation, which is shown to be the value function of our control problem.

Full work available at URL: https://arxiv.org/abs/1311.1063

Recommendations

• Backward stochastic differential equations and optimal control of marked point processes
• Backward stochastic differential equations associated to jump Markov processes and applications
• scientific article; zbMATH DE number 1066316
• Backward SDEs and infinite horizon stochastic optimal control
• Optimal control of piecewise deterministic Markov processes: a BSDE representation of the value function

zbMATH Keywords

backward stochastic differential equationsoptimal control problemsmarked point processessemi-Markov processes

Mathematics Subject Classification ID

Stochastic ordinary differential equations (aspects of stochastic analysis) (60H10) Optimal stochastic control (93E20)

Cites Work

• Multivariate point processes: predictable projection, Radon-Nikodym derivatives, representation of martingales
• Title not available (Why is that?)
• Title not available (Why is that?)
• Optimal control of piecewise deterministic markov process
• Necessary Conditions for Optimal Control of Stochastic Systems with Random Jumps
• Continuous-time Markov decision processes. Theory and applications
• Optimal trade execution in illiquid markets
• Backward stochastic differential equations and integral-partial differential equations
• Backward stochastic differential equations with jumps and related nonlinear expectations
• Bounded solutions to backward SDEs with jumps for utility optimization and indifference hedging
• Point processes and queues. Martingale dynamics
• Title not available (Why is that?)
• Markov-Renewal Programming. I: Formulation, Finite Return Models
• Title not available (Why is that?)
• Reflected and doubly reflected BSDEs with jumps: a priori estimates and comparison
• A numerical algorithm for fully nonlinear HJB equations: an approach by control randomization
• Discrete time approximation of fully nonlinear HJB equations via BSDEs with nonpositive jumps
• Backward stochastic differential equations and optimal control of marked point processes
• Backward SDEs with constrained jumps and quasi-variational inequalities
• Backward stochastic differential equation with random measures
• \(L^p\) solution of backward stochastic differential equations driven by a marked point process
• Backward stochastic differential equations associated to jump Markov processes and applications
• Backward stochastic differential equation driven by a marked point process: an elementary approach with an application to optimal control
• Optimal high-frequency trading with limit and market orders
• Bellman inequalities in markov decision deterministic drift processes
• Continuous average control of piecewise deterministic Markov processes
• Existence of Optimal Stochastic Control Laws
• Dynamic Programming Conditions for Partially Observable Stochastic Systems
• Martingale conditions for the optimal control of continuous time stochastic systems
• Optimal Control of Jump Processes
• Title not available (Why is that?)
• Martingales on Jump Processes. I: Representation Results
• Continuously Discounted Markov Decision Model with Countable State and Action Space
• MDP algorithms for portfolio optimization problems in pure jump markets
• Controlled jump processes
• Piecewise deterministic Markov control processes with feedback controls and unbounded costs
• Generalized Bellman-Hamilton-Jacobi optimality conditions for a control problem with a boundary condition
• A dynamic programming algorithm for the optimal control of piecewise deterministic Markov processes
• Second order BSDEs with jumps: existence and probabilistic representation for fully-nonlinear PIDEs
• Linear programming algorithms for semi-Markovian decision processes
• Title not available (Why is that?)
• Title not available (Why is that?)
• On Reducing a Jump Controllable Markov Model to a Model with Discrete Time
• Title not available (Why is that?)
• Necessary and sufficient optimality conditions for control of piecewise deterministic markov processes
• Constrained BSDEs representation of the value function in optimal control of pure jump Markov processes
• Continuous Time Markovian Sequential Control Processes
• Necessary and Sufficient Conditions for Optimal Control of Semi-Markov Jump Processes
• Second-order BSDEs with jumps: formulation and uniqueness
• Existence and uniqueness for backward stochastic differential equations driven by a random measure, possibly non quasi-left continuous

Cited In (14)

• Dynamic programming for semi-Markov modulated SDEs
• Backward SDEs and infinite horizon stochastic optimal control
• The identification problem for BSDEs driven by possibly non-quasi-left-continuous random measures
• Backward stochastic differential equation driven by a marked point process: an elementary approach with an application to optimal control
• Special weak Dirichlet processes and BSDEs driven by a random measure
• Optimal control for stochastic Volterra equations with multiplicative Lévy noise
• Backward SDEs for optimal control of partially observed path-dependent stochastic systems: A control randomization approach
• \(L^p\) solution of backward stochastic differential equations driven by a marked point process
• Weak Dirichlet processes with jumps
• Backward stochastic differential equations associated to jump Markov processes and applications
• Optimal control of piecewise deterministic Markov processes: a BSDE representation of the value function
• Backward stochastic differential equations and optimal control of marked point processes
• On the monotone stability approach to BSDEs with jumps: extensions, concrete criteria and examples
• Constrained BSDEs representation of the value function in optimal control of pure jump Markov processes

This page was built for publication: Optimal control of semi-Markov processes with a backward stochastic differential equations approach

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q525049)