Optimal control of semi-Markov processes with a backward stochastic differential equations approach

DOI10.1007/S00498-016-0181-6MaRDI QIDQ525049zbMATH OpenOpenAlexDBLPWikidataFDO

Authors Elena Bandini, F. Confortola

Publication date 28 April 2017

Published in MCSS. Mathematics of Control, Signals, and Systems (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1311.1063

zbMATH Keywords

backward stochastic differential equationsoptimal control problemsmarked point processessemi-Markov processes

Mathematics Subject Classification ID

Stochastic ordinary differential equations (aspects of stochastic analysis) (60H10) Optimal stochastic control (93E20)

Abstract: In the present work we employ, for the first time, backward stochastic differential equations (BSDEs) to study the optimal control of semi-Markov processes on finite horizon, with general state and action spaces. More precisely, we prove that the value function and the optimal control law can be represented by means of the solution of a class of BSDEs driven by a semi-Markov process or, equivalently, by the associated random measure. The peculiarity of the semi-Markov framework, with respect to the pure jump Markov case, consists in the proof of the relation between BSDE and optimal control problem. This is done, as usual, via the Hamilton-Jacobi-Bellman (HJB) equation, which however in the semi-Markov case is characterized by an additional differential term $partia{l}_a$. Taking into account the particular structure of semi-Markov processes we rewrite the HJB equation in a suitable integral form which involves a directional derivative operator $D$ related to $partia{l}_a$. Then, using a formula of Ito type tailor-made for semi-Markov processes and the operator $D$, we are able to prove that the BSDE provides the unique classical solution to the HJB equation, which is shown to be the value function of our control problem.

Recommendations

• Backward stochastic differential equations and optimal control of marked point processes
• Backward stochastic differential equations associated to jump Markov processes and applications
• scientific article; zbMATH DE number 1066316
• Backward SDEs and infinite horizon stochastic optimal control
• Optimal control of piecewise deterministic Markov processes: a BSDE representation of the value function

Cites work

• scientific article; zbMATH DE number 425394 (Why is no real title available?)
• scientific article; zbMATH DE number 3842850 (Why is no real title available?)
• scientific article; zbMATH DE number 4208812 (Why is no real title available?)
• scientific article; zbMATH DE number 3793150 (Why is no real title available?)
• scientific article; zbMATH DE number 17494 (Why is no real title available?)
• scientific article; zbMATH DE number 3490831 (Why is no real title available?)
• scientific article; zbMATH DE number 3338194 (Why is no real title available?)
• scientific article; zbMATH DE number 3361677 (Why is no real title available?)
• A dynamic programming algorithm for the optimal control of piecewise deterministic Markov processes
• A numerical algorithm for fully nonlinear HJB equations: an approach by control randomization
• Backward SDEs with constrained jumps and quasi-variational inequalities
• Backward stochastic differential equation driven by a marked point process: an elementary approach with an application to optimal control
• Backward stochastic differential equation with random measures
• Backward stochastic differential equations and integral-partial differential equations
• Backward stochastic differential equations and optimal control of marked point processes
• Backward stochastic differential equations associated to jump Markov processes and applications
• Backward stochastic differential equations with jumps and related nonlinear expectations
• Bellman inequalities in markov decision deterministic drift processes
• Bounded solutions to backward SDEs with jumps for utility optimization and indifference hedging
• Constrained BSDEs representation of the value function in optimal control of pure jump Markov processes
• Continuous Time Markovian Sequential Control Processes
• Continuous average control of piecewise deterministic Markov processes
• Continuous-time Markov decision processes. Theory and applications
• Continuously Discounted Markov Decision Model with Countable State and Action Space
• Controlled jump processes
• Discrete time approximation of fully nonlinear HJB equations via BSDEs with nonpositive jumps
• Dynamic Programming Conditions for Partially Observable Stochastic Systems
• Existence and uniqueness for backward stochastic differential equations driven by a random measure, possibly non quasi-left continuous
• Existence of Optimal Stochastic Control Laws
• Generalized Bellman-Hamilton-Jacobi optimality conditions for a control problem with a boundary condition
• Linear programming algorithms for semi-Markovian decision processes
• MDP algorithms for portfolio optimization problems in pure jump markets
• Markov-Renewal Programming. I: Formulation, Finite Return Models
• Martingale conditions for the optimal control of continuous time stochastic systems
• Martingales on Jump Processes. I: Representation Results
• Multivariate point processes: predictable projection, Radon-Nikodym derivatives, representation of martingales
• Necessary Conditions for Optimal Control of Stochastic Systems with Random Jumps
• Necessary and Sufficient Conditions for Optimal Control of Semi-Markov Jump Processes
• Necessary and sufficient optimality conditions for control of piecewise deterministic markov processes
• On Reducing a Jump Controllable Markov Model to a Model with Discrete Time
• Optimal Control of Jump Processes
• Optimal control of piecewise deterministic markov process
• Optimal high-frequency trading with limit and market orders
• Optimal trade execution in illiquid markets
• Piecewise deterministic Markov control processes with feedback controls and unbounded costs
• Point processes and queues. Martingale dynamics
• Reflected and doubly reflected BSDEs with jumps: a priori estimates and comparison
• Second order BSDEs with jumps: existence and probabilistic representation for fully-nonlinear PIDEs
• Second-order BSDEs with jumps: formulation and uniqueness
• \(L^p\) solution of backward stochastic differential equations driven by a marked point process

Cited in

(14)

• Optimal control of piecewise deterministic Markov processes: a BSDE representation of the value function
• Backward SDEs for optimal control of partially observed path-dependent stochastic systems: A control randomization approach
• On the monotone stability approach to BSDEs with jumps: extensions, concrete criteria and examples
• Backward SDEs and infinite horizon stochastic optimal control
• \(L^p\) solution of backward stochastic differential equations driven by a marked point process
• Backward stochastic differential equations and optimal control of marked point processes
• The identification problem for BSDEs driven by possibly non-quasi-left-continuous random measures
• Weak Dirichlet processes with jumps
• Backward stochastic differential equation driven by a marked point process: an elementary approach with an application to optimal control
• Special weak Dirichlet processes and BSDEs driven by a random measure
• Backward stochastic differential equations associated to jump Markov processes and applications
• Constrained BSDEs representation of the value function in optimal control of pure jump Markov processes
• Optimal control for stochastic Volterra equations with multiplicative Lévy noise
• Dynamic programming for semi-Markov modulated SDEs

This page was built for publication: Optimal control of semi-Markov processes with a backward stochastic differential equations approach

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q525049)