Analysis of an identification algorithm arising in the adaptive estimation of Markov chains (Q1262282)

The paper is devoted to the adaptive estimation problem for partially observable finite-state Markov chains. The authors describe an algorithm which utilizes the recursive equation characterizing the conditional distribution of the state of the Markov chain, given the past observations. Several important analytical properties of this algorithm are studied. In particular, an interesting connection is established with the ordinary differential equation method for stochastic approximations. This allows to give a deep analysis of the algorithms suggested here. We find in the paper clearly formulated statements as well as their detailed proofs. Some useful related topics are also discussed.