Asymptotic bias of stochastic gradient search (Q1704136): Difference between revisions

There is an investigation on the asymptotic behavior of biased stochastic gradient search. The following algorithm is analyzed: \({\theta}_{n+1}\) = \({\theta}_{n}\)- \({\alpha}_{n}({\nabla}f({\theta}_{n})+{\xi}_{n})\) , \(n{\geq}0\). Under a set of assumptions regarding the step-size sequence, the noise and the objective function f , the convergence of the algorithm iterates to a neighborhood of the set of minima, is proved. Upper bounds on the radius of the vicinity are obtained. The results are local, they hold only in case the stated algorithm is stable. The proofs are relaying on the chain-recurrence, Yomdin theorem and Lojasiewicz inequalities. Further , the obtained results are applied to stochastic gradient algorithms with Markovian dynamics and to the asymptotic analysis of a policy-gradient search algorithm for average-cost Markov decision problems. Global versions of the results are presented in the Appendix A and Appendix B of the paper. There is stipulated that an extended version of this article is available at \url{arXiv:1709.00291}.