Adaptive policies for time-varying stochastic systems under discounted criterion (Q1397033)

The authors consider a discrete-time controlled Markov system whose evolution is described by the equation \(x_{n+1}= G_n(x_n, a_n,\xi_n)\), \(n= 0,1,\dots\), where the system states \(x_n\) and controls \(a_n\) are elements of Borel spaces and \(\{\xi_n\}\) is a sequence of observable i.i.d. random vectors with unknown distribution. Assuming the convergence of \(G_n\) and estimating the unknown distribution density of \(\xi_n\), an asymptotically optimal control policy for the limit control system is constructed.