An efficient numerical algorithm for solving data driven feedback control problems (Q2219806)

From MaRDI portal
scientific article
Language Label Description Also known as
English
An efficient numerical algorithm for solving data driven feedback control problems
scientific article

    Statements

    An efficient numerical algorithm for solving data driven feedback control problems (English)
    0 references
    0 references
    21 January 2021
    0 references
    Consider a stochastic differential equation (SDE) \(dX_t=b(t,X_t,u_t) dt+\sigma(t,X_t,u_t) dW_t\) over a deterministic time interval \(t \in [0,T]\) with \(X_0=\xi\) where \(X_t \in\mathbb{R}^d\) is the controlled state process, \(u_t\in U \subseteq \mathbb{R}^m\) is the control process, \(W_t\) is a \(d\)-dimensional standard Brownian motion, \(\xi\) is a random variable independent of \(W_t\). Observation process \(M_t\) is described by SDE \(d M_t = g(X_t)+dB_t\) with \(M_0=0\) where \(B_t\) is an \(\ell\)-dimensional standard Brownian motion independent of \(W_t\). The cost functional is \(J(u^M)=\mathbb{E}\left(\int_0^T f(t,X_t,u^M_t) dt + h(X_T)\right)\) where \(u^M_t\) is a control process based on observations of \(M_t\), \(\mathbb{E}\) is the sign of mathematical expectation. The goal of the control is to find \(u^*\) such that \(J(u^*)=\inf_{u^M} J(u^M)\). The optimal control process is determined numerically using gradient descent method according to the algorithm \(u_t^{l+1,M}=u_t^{l,M}-\rho J'_u(u_t^{l,M})\), \(l=0,1.\dots\), where \(\rho\) is the step-size for the gradient. Here \(J'_u(u_t^{l,M})\) is a Gateaux derivative which can be determined by solving appropriate system of forward/backward SDE and is computed using Monte-Carlo simulations. Three illustrative examples which demonstrate efficiency of computational framework are presented.
    0 references
    stochastic optimal control
    0 references
    nonlinear filtering
    0 references
    data driven
    0 references
    maximum principle
    0 references
    stochastic optimization
    0 references
    0 references
    0 references
    0 references
    0 references

    Identifiers

    0 references
    0 references
    0 references
    0 references
    0 references
    0 references
    0 references