Discretized Maximum Likelihood and Almost Optimal Adaptive Control of Ergodic Markov Models (Q4388920)

Three types of controlled ergodic Markov processes are considered: A completely observed discrete-time process, a completely observed diffusion, and a partially observed discrete-time process. The partial observation consists of complete observation on a fixed recurrent set and noisy observation on its complement. In each case an almost self-optimizing control is constructed from a family of estimates using a finite parameter discretization and a finite family of almost optimal ergodic controls by a randomized certainty equivalence method. The almost optimality property is obtained via a continuity property of the information of a model for one parameter value with respect to another. Various generalizations are immediate: The discrete time model can be modified to include the recurrence model of [\textit{L. Stettner}, J. Appl. Math. Optimization 27, 161-177 (1993; Zbl 0769.93084)], the diffusion model can be generalized to that of [\textit{T. E. Duncan}, \textit{B. Pasik-Duncan} and \textit{L. Stettner}, J. Optimization Theory Appl. 81, 509-531 (1994; Zbl 0801.60048)] to include processes that solve stochastic differential equations with delays. The partial observation structure can be modified to noisy observation everywhere, if there is a sequence of random times such that the process at these times is a family of i.i.d. variables.