Adaptive control of constrained finite Markov chains (Q1298214)

The paper presents an approach for the adaptive control of constrained finite Markov chains. The control of finite Markov chains is a well-studied problem and approaches to adaptive control of finite Markov chains have been developed for more than 30 years. The contribution of this paper is claimed to be a novel algorithm for the control of constrained Markov chains with unknown transition probabilities. The paper uses Lagrange multipliers to reduce the problem to an unconstrained problem and proposes stochastic approximation to derive the adaptive control policy. Both ideas are hardly novel. The ideas in the paper are very obscured by a multitude of definitions and theorems most of them are well-known and are stated without reference to their sources. The paper states convergence results but the constrained problem may have no solution and the convergence may be too slow to have practical value. The paper gives no discussion of the complexity of the algorithm or mentions any experience in applying the algorithm to specific examples or real-life applications.