Self-tuning control of diffusions without the identifiability condition

DOI10.1007/BF00939938MaRDI QIDQ912052zbMATH OpenOpenAlexFDO

Authors Vivek Borkar

Publication date 1991

Published in Journal of Optimization Theory and Applications (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1007/bf00939938

zbMATH Keywords

identifiability conditionKumar-Becker-Lin schemeself-tuning scheme

Mathematics Subject Classification ID

Control/observation systems governed by partial differential equations (93C20) Adaptive control/observation systems (93C40) Estimation and detection in stochastic control theory (93E10)

Review text

The self-tuning method of adaptive control for diffusions consists of estimating the unknown parameter on line and using its current estimate as the true parameter for selection of the control at each time. The a.s. optimality of this scheme for the ergodic or long-run average criterion can be established under an identifiability condition on the system, but may fail otherwise. We present a modified self-tuning scheme along the lines of the Kumar- Becker-Lin scheme for Markov chains [see \textit{P. R. Kumar} and \textit{A. Becker}, IEEE Trans. Autom. Control AC-27, 137-146 (1982; Zbl 0471.93069); \textit{P. R. Kumar} and \textit{W. Lin}, ibid., 765-774 (1982; Zbl 0488.93036)] and prove its a.s. optimality. Several heuristic issues related to this scheme are also discussed.

Recommendations

• scientific article; zbMATH DE number 409876
• Ergodic control of multidimensional diffusions. II: Adaptive control
• Almost self-optimizing strategies for the adaptive control of diffusion processes
• scientific article; zbMATH DE number 4068727
• scientific article; zbMATH DE number 778101

Cites work

• A new family of optimal adaptive controllers for Markov chains
• Adaptive control of Markov chains, I: Finite parameter set
• Controlled diffusions with boundary-crossing costs
• Ergodic Control of Multidimensional Diffusions I: The Existence Results
• Ergodic control of multidimensional diffusions. II: Adaptive control
• Existence of Optimal Strategies Based on Specified Information, for a Class of Stochastic Decision Problems
• scientific article; zbMATH DE number 3780265 (Why is no real title available?)
• scientific article; zbMATH DE number 3599198 (Why is no real title available?)
• Identification and Adaptive Control of Markov Chains
• ON STRONG SOLUTIONS AND EXPLICIT FORMULAS FOR SOLUTIONS OF STOCHASTIC INTEGRAL EQUATIONS
• Optimal adaptive controllers for unknown Markov chains
• Parameter estimation in continuous-time stochastic processes
• Strong consistency of a modified maximum likelihood estimator for controlled Markov chains
• The Kumar-Becker-Lin scheme revisited

Cited in

(8)

• Almost self-optimizing strategies for the adaptive control of diffusion processes
• On the ergodic and the adaptive control of stochastic differential delay systems
• The Kumar-Becker-Lin scheme revisited
• scientific article; zbMATH DE number 4068727 (Why is no real title available?)
• scientific article; zbMATH DE number 409876 (Why is no real title available?)
• Stochastic \varepsilon-Optimal Linear Quadratic Adaptation: An Alternating Controls Policy
• Stochastic adaptive control with small observation noise
• Ergodic control of multidimensional diffusions. II: Adaptive control

This page was built for publication: Self-tuning control of diffusions without the identifiability condition

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q912052)