Discretized Maximum Likelihood and Almost Optimal Adaptive Control of Ergodic Markov Models

DOI10.1137/S0363012996298369MaRDI QIDQ4388920zbMATH OpenFDO

Authors T. E. Duncan, B. Pasik-Duncan, Łukasz Stettner

Publication date 10 May 1998

Published in SIAM Journal on Control and Optimization (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1137/S0363012996298369

zbMATH Keywords

adaptive controlalmost optimal adaptive controlcontrolled Markov processesergodic controlMarkov processes

Mathematics Subject Classification ID

Discrete-time Markov processes on general state spaces (60J05) Markov processes: estimation; hidden Markov models (62M05) Adaptive control/observation systems (93C40) Stochastic learning and adaptive control (93E35)

Review text

Three types of controlled ergodic Markov processes are considered: A completely observed discrete-time process, a completely observed diffusion, and a partially observed discrete-time process. The partial observation consists of complete observation on a fixed recurrent set and noisy observation on its complement. In each case an almost self-optimizing control is constructed from a family of estimates using a finite parameter discretization and a finite family of almost optimal ergodic controls by a randomized certainty equivalence method. The almost optimality property is obtained via a continuity property of the information of a model for one parameter value with respect to another. Various generalizations are immediate: The discrete time model can be modified to include the recurrence model of [\textit{L. Stettner}, J. Appl. Math. Optimization 27, 161-177 (1993; Zbl 0769.93084)], the diffusion model can be generalized to that of [\textit{T. E. Duncan}, \textit{B. Pasik-Duncan} and \textit{L. Stettner}, J. Optimization Theory Appl. 81, 509-531 (1994; Zbl 0801.60048)] to include processes that solve stochastic differential equations with delays. The partial observation structure can be modified to noisy observation everywhere, if there is a sequence of random times such that the process at these times is a family of i.i.d. variables.

Recommendations

• A first order approximation forthe convergence of distributionsof the cox processes with
• Ergodic and adaptive control of hidden Markov models
• scientific article; zbMATH DE number 756533
• Adaptive control of a partially observed discrete time Markov process

Cited in

(4)

• Ergodic and adaptive control of hidden Markov models
• Maximum Likelihood Estimation of Discrete Control Processes
• A first order approximation forthe convergence of distributionsof the cox processes with
• scientific article; zbMATH DE number 756533 (Why is no real title available?)

This page was built for publication: Discretized Maximum Likelihood and Almost Optimal Adaptive Control of Ergodic Markov Models

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4388920)