Adaptive control of constrained finite Markov chains

DOI10.1016/S0005-1098(98)00219-2MaRDI QIDQ1298214zbMATH OpenOpenAlexFDO

Authors Alexander S. Poznyak, K. Najim

Publication date 20 December 1999

Published in Automatica (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1016/s0005-1098(98)00219-2

zbMATH Keywords

adaptive controlconstrained Markov chainsfinite Markov chainsstochastic approximation

Mathematics Subject Classification ID

Adaptive control/observation systems (93C40) Stochastic learning and adaptive control (93E35)

Review text

The paper presents an approach for the adaptive control of constrained finite Markov chains. The control of finite Markov chains is a well-studied problem and approaches to adaptive control of finite Markov chains have been developed for more than 30 years. The contribution of this paper is claimed to be a novel algorithm for the control of constrained Markov chains with unknown transition probabilities. The paper uses Lagrange multipliers to reduce the problem to an unconstrained problem and proposes stochastic approximation to derive the adaptive control policy. Both ideas are hardly novel. The ideas in the paper are very obscured by a multitude of definitions and theorems most of them are well-known and are stated without reference to their sources. The paper states convergence results but the constrained problem may have no solution and the convergence may be too slow to have practical value. The paper gives no discussion of the complexity of the algorithm or mentions any experience in applying the algorithm to specific examples or real-life applications.

Recommendations

• scientific article; zbMATH DE number 1461223
• Penalty function and adaptive control of constrained finite Markov chains
• Adaptive control of constrained Markov chains: Criteria and policies

Cited in

(8)

• On the Milito-Cruz adaptive control scheme for Markov chains
• scientific article; zbMATH DE number 3849083 (Why is no real title available?)
• scientific article; zbMATH DE number 4203409 (Why is no real title available?)
• scientific article; zbMATH DE number 4123661 (Why is no real title available?)
• Asymptotically Efficient Adaptive Choice of Control Laws inControlled Markov Chains
• scientific article; zbMATH DE number 1461223 (Why is no real title available?)
• Dr. Alexander Semionovich Poznyak Gorbatch: biography
• Adaptive policy for two finite Markov chains zero-sum stochastic game with unknown transition matrices and average payoffs

This page was built for publication: Adaptive control of constrained finite Markov chains

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1298214)