Asymptotic properties of quasi-optimal algorithms of stochastic approximation under unknown density of noise (Q1287252)

The author considers a classical problem of stochastic approximation, where it is required to estimate the root \(x^*\) of the unknown function \(f\) from the successively arriving observations \(y_n= f(x_{n-1})+ \xi_n\). Here, \(\xi_n\) are independent identically distributed random noisy variables with \(E\xi_n= O\) and unknown density \(p(\cdot)\). An optimal algorithm can be realized as a two-step procedure \[ x_{n+1}= x_n- \gamma n^{-\alpha} \varphi(y_{n+1}),\quad \overline x_{n+1}= \overline x_n- n^{-1} (\overline x_n- x_n), \] where \(\gamma>0\), \(1/2<\alpha< 1\), \(\varphi_{\text{opt}}(y)= p'(y)/p(y)\). The function \(\varphi_{\text{opt}}\) is approximated in some \(N\)-parametric family \(\varphi\) of functions \(\varphi= \varphi(\cdot,\theta)\), \(\theta\in \Theta\subset \mathbb{R}^N\). Quasi-optimal estimates of the root \(x^*\) are constructed with the help of an algorithm recurrent in \(\theta\) for approximation of the function \(\varphi_{\text{opt}}\) in the family \(\varphi\) under the observations of \(y_n\). The points \(x_{n-1}\) of measurement are generated by the two-step procedure mentioned above with the help of the current approximation of \(\varphi(y,\theta_{n-1})\) instead of \(\varphi_{\text{opt}}\), and the averaged estimates \(\overline x_n\) must have the convergence rate \[ \sqrt n(\overline x_n- x_n)\sim N(O,V(\varphi(\cdot,\theta^*, p)). \] Here \(\theta^*\) is the point where the minimum \(\min_{\theta\in\Theta}{\mathcal I}(\varphi(\cdot,\theta),p)\) is achieved, and \({\mathcal I}(\varphi,p)= \int(\varphi- 2\varphi'(y))p dy\), \(V\) is the asymptotic variance.