An efficient numerical algorithm for solving data driven feedback control problems (Q2219806)

Consider a stochastic differential equation (SDE) \(dX_t=b(t,X_t,u_t) dt+\sigma(t,X_t,u_t) dW_t\) over a deterministic time interval \(t \in [0,T]\) with \(X_0=\xi\) where \(X_t \in\mathbb{R}^d\) is the controlled state process, \(u_t\in U \subseteq \mathbb{R}^m\) is the control process, \(W_t\) is a \(d\)-dimensional standard Brownian motion, \(\xi\) is a random variable independent of \(W_t\). Observation process \(M_t\) is described by SDE \(d M_t = g(X_t)+dB_t\) with \(M_0=0\) where \(B_t\) is an \(\ell\)-dimensional standard Brownian motion independent of \(W_t\). The cost functional is \(J(u^M)=\mathbb{E}\left(\int_0^T f(t,X_t,u^M_t) dt + h(X_T)\right)\) where \(u^M_t\) is a control process based on observations of \(M_t\), \(\mathbb{E}\) is the sign of mathematical expectation. The goal of the control is to find \(u^*\) such that \(J(u^*)=\inf_{u^M} J(u^M)\). The optimal control process is determined numerically using gradient descent method according to the algorithm \(u_t^{l+1,M}=u_t^{l,M}-\rho J'_u(u_t^{l,M})\), \(l=0,1.\dots\), where \(\rho\) is the step-size for the gradient. Here \(J'_u(u_t^{l,M})\) is a Gateaux derivative which can be determined by solving appropriate system of forward/backward SDE and is computed using Monte-Carlo simulations. Three illustrative examples which demonstrate efficiency of computational framework are presented.