Approximations for discrete-time adaptive control: Construction of \(\varepsilon\)-optimal controls (Q1176543)

The following problem of stochastic optimal control is considered. To minimize the functional \(G(u)=M\left\{\sum^{N-1}_{n=0}g_ 1(x_ n,u_ n)+g_ 2(x_ N)\right\}\to\inf\) under the conditions \[ x_{n+1}=f_ 1(x_ n,u_ n)+\theta f_ 2(x_ n,u_ n)+\sigma(x_ n,u_ n)w_{n+1},\qquad n=0,\dots,N-1, \] where \(x_ 0\) is the initial condition with density of distribution \(p_ 0(x_ 0)\), \(\{w_ n\}\) is the Gaussian white noise, \(\theta\) is a random parameter with density of distribution \(p(\theta)\), \(u_ n\) is a control parameter taking values in a compact set \(u\). The observation process \(y_ n=c(x_{n+1})+v_ n\), where \(\{v_ n\}\) is the Gaussian white noise, random variables \(x_ 0\), \(\{w_ n\}\), \(\{v_ n\}\) are mutually independent. It is required to determining an optimal control \(u^*_ n=u_ n(y^ n,u^{n-1})\), where \(y^ n=(y_ 0,y_ 1,\dots,y_ n)\), \(u^{(n-1)}=(u_ 0,\dots,u_{n- 1})\). The exact determination of optimal control \(u_ n^*\), \(n=0,\dots,N-1\) is a fairly complicated problem, therefore an approximate method of computation the \(\tau\)-optimal control \(u^ \tau\) for which \(G(u^ \tau)\leq\inf G(u)+\tau\), is suggested. For this purpose, the original problem is replaced by an approximating problem, then for the approximating problem the optimal control can be obtained. For the case, when \(| f_ 1(x,u)|\leq| x|-A\), \(| f_ 2(x,u)|\leq B\), there are the limits \(\lim_{| x|\to\infty}f_ 2(x,u)\), \(0<c\leq\sigma(x,u)\leq d\), \(\lim_{| x|\to\infty}\sigma(x,u)\), \(| g_ 1(x,u)|\leq c_ 2\times e^{| x|}\), \(| g_ 2(x)|\leq c_ 3e^{| x|}\) and functions. \(f_ 1\), \(f_ 2\), \(\sigma\), \(g_ 1\), \(g_ 2\) are continuous functions of their arguments. It is shown that, for a given \(\tau>0\) there exists an approximating problem, such that the optimal solution for the latter is a \(\tau\)-optimal solution for the original problem.