An algorithm for solving a stochastic control problem

DOI10.1080/07362999608809455MaRDI QIDQ4342975zbMATH OpenOpenAlexFDO

Authors Laurent Mazliak

Publication date 24 August 1997

Published in Stochastic Analysis and Applications (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1080/07362999608809455

zbMATH Keywords

algorithmconvex functionmaximum principleweak convergence

Mathematics Subject Classification ID

Optimal stochastic control (93E20)

Review text

Some years ago, Sakawa-Shindo and Bonnans have introduced an algorithm for the solution of a deterministic control problem. This algorithm is based on the maximum principle or Pontryagin's principle. The major idea of these authors was to transform the Hamiltonian function into a strictly convex function. Kushner and Haussmann extended the maximum principle to the case of stochastic diffusions. This principle is difficult to apply in general because the equation for the adjoint process contains an unknown quantity. We are first interested in the control of a stochastic diffusion process for which the diffusion matrix is given. In this case, the equations for the adjoint process are considerably simplified and we are able to extend the algorithm to it. In a first part, we set the problem, prove some boundedness properties for the adjoint process and formally define the algorithm. We then prove that the algorithm is indeed well defined. In a second part, we study the convergence problem and prove a weak convergence result in a suitable convexity context. Then, we deal with a more general case where the diffusion coefficient is a random matrix, define the algorithm and obtain a convergence result for it.

Recommendations

• On the convergence of the Sakawa-Shindo algorithm in stochastic control
• An Algorithm for Infinite Dimensional Stochastic Control Problems
• A successive approximation algorithm for stochastic control problems
• scientific article; zbMATH DE number 953309

Cites work

• scientific article; zbMATH DE number 3778463 (Why is no real title available?)
• scientific article; zbMATH DE number 3505981 (Why is no real title available?)
• Necessary Conditions for Continuous Parameter Stochastic Optimization Problems
• On an Algorithm for Optimal Control Using Pontryagin’s Maximum Principle
• On global convergence of an algorithm for optimal control

Cited in

(9)

• A stochastic gradient type algorithm for closed-loop problems
• A modified MSA for stochastic control problems
• Policy iteration algorithm for singular controlled diffusion processes
• Stochastic optimal control to a nonlinear differential game
• scientific article; zbMATH DE number 425620 (Why is no real title available?)
• On the convergence of the Sakawa-Shindo algorithm in stochastic control
• scientific article; zbMATH DE number 4005971 (Why is no real title available?)
• On the relaxed mean-field stochastic control problem
• An Algorithm for Infinite Dimensional Stochastic Control Problems

This page was built for publication: An algorithm for solving a stochastic control problem

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4342975)