Controlled diffusion processes on infinite horizon with the overtaking criterion (Q1098487)

Let x(t,u) be a diffusion process depending on a control u. The paper is concerned with the minimization of functionals of the form \[ C_ T(u)=E\int^{T}_{0}f(x(t),u(t),t)dt,\quad as\quad T\to \infty, \] where f is a given function. In many cases \(C_ T(u)\) diverges for any choice of u, and the introduction of discounting factors is not justified. Thus the author studies the problem of minimizing \[ J(u)=\liminf (1/T)C_ T(u)\quad as\quad T\to \infty, \] which describes the minimal growth rate of \(C_ T(u)\) (similar problems with lim sup are also considered). The overtaking criterion is discussed for this infinite horizon optimal control problem. The aim of the paper is twofold: i) to achieve a better understanding of the Bellman equation arising in the optimal control problem (in particular, to give a cost interpretation of the solution of the Bellman equation), ii) to find conditions, in terms of the solutions of the Bellman equation, for the existence of overtaking optimal controls, or minimal time average controls, and to characterize them using the Bellman equation. Control problems with a non-integrable discount factor are also studied. The author considers both cases, where the diffusion is in R d, or it takes place in a bounded domain with reflections from the boundary.