Numerical comparison of controls and verification of optimality for stochastic control problems (Q1586818)

The purpose of this paper is to provide a numerical method for comparing the efficiency of various controls and to present a verification approach to the solution to stochastic control problems. A Linear Programming (LP) formulation involving the moments of the stationary distribution of the process is used. Two approaches were adopted. The first was to specify the control for the process and run the LP to obtain various characteristics of the controlled process including the cost associated with the process. Doing this with several controls allows a comparison of the costs. The second approach was to specify a candidate for the optimal control and derive a modified LP which all controlled processes satisfied and which the candidate control satisfied with tightness for particular constraints. This differs from dynamic programming in which one guesses the value function, verifies that it is a solution to the Hamilton-Jacobi-Bellman equation, and as a consequence determines the optimal control. For numerical computation, the problem is reduced to finite dimensions by considering only a finite number of moments. The proposed approach deals directly with the stochastic process of the control problem rather than with some approximating process.