Optimal, recursive procedures of identification (Q1105556)

The paper concerns the parameter estimation of a stochastic process. We consider recursive estimating procedures based on the idea of stochastic approximation: (1) \(p_{s+1}=p_ s-\mu_ sF_{s+1}(Y_{s+1},p_ s)\), where \(\{\mu_ s\}\) is a sequence of positivd to the other ones when \(\beta\uparrow 1.\) Recently a variant of the discounted multi-armed bandit problem has been described by \textit{P. P. Varaiya, J. C. Walrand} and \textit{C. Buyukkoc} [IEEE Trans. Autom. Control 30, 426-439 (1985; Zbl 0566.90096)]. In this variant the k machines are represented by independent Markov chains. It is motivated by priority scheduling in networks. The relations of this variant with the third formulation given above is described.