Existence of Markov Controls and Characterization of Optimal Markov Controls (Q4388926)

Consider a controlled martingale problem under the requirement that \[ f(X(t))- \int^t_0 Af(X(s), u(s)) ds \] be a martingale for \(f\in{\mathcal D}(A)\), where \(u\) is the control process. The existenc of an optimal Markov control has been proved by using the Krylov selection theorem, which, however, does not characterize the control [cf. \textit{U. G. Haussmann} and \textit{J. P. Lepeltier}, SIAM J. Control Optimization 28, 851-902 (1990; Zbl 0712.49013)]. In this paper, under general conditions for the generator \(A\), it is shown that there exists a so-called relaxed solution having Markov controls which has the same cost as the original solution. It is then shown that the original control problem can be reformulated as a linear programming (LP) problem over an appropriate space of measures. With this LP approach, characterization as well as existence of optimal Markov controls can directly follow. The equivalent LP formulation (and expressions for minimization) under a variety of cost criteria is developed in detail.