Adaptive control of partially observable homogeneous processes with independent values (Q1895477)

This paper deals with the adaptive control problems \((X, Z, U, M^*, L^*, Q^*, W)\) under the following assumptions: 1) The class \(M^*\) of controlled objects is the entire class of homogeneous processes with independent values with state space \(X= \{0, 1\}\) and control space \(U= \{1, 2\}\) defined by a single function \(\mu= \{\mu(x|u); x\in X, u\in U\}\), where the probability \(\mu(x|u)\geq 0\) satisfies \(\mu(1|u)+ \mu(0|u)= 1\). 2) The class \(L^*\) of observation processes is a certain class of homogeneous ones with observation space \(Z= \{0, 1\}\) defined by \(\lambda= \{\lambda_t(z_t|x^t, z^{t- 1}, u^{t- 1}): t\geq 1\}\) with \(x^t= (x_t, \dots, x_1)\), \(z^t= (z_t, \dots, z_1)\), and \(u^{t- 1}= (u_{t- 1},\dots, u_0)\) and \[ \lambda_t(z_t|x^t, z^{t- 1}, u^{t- 1})= \begin{cases} 1/2,\quad & t< m,\\ \lambda_t(z_t|x_t,\dots, x_{t- m+ 1}),\quad & t\geq m,\end{cases} \] where \(m\) is a natural number, and \(\lambda(z|y_m,\dots, y_1)\geq 0\) satisfies \[ \lambda(1|y_m,\dots, y_1)+ \lambda(0|y_m,\dots, y_1)= 1. \] 3) The class \(Q^*\) of general strategies can be described by a sequence \[ \nu= \{\nu_0(u_0), \nu_t(u_t|z^t, u^{t- 1}): t\geq 1\}, \] where \(\nu_t(u_t|z^t, u^{t- 1})\) is the probability of application of the control \(u_t\) at time \(t\) provided that the observation \(u^{t- 1}= (u_{t- 1},\dots u_0)\) were already used. 4) For any triple \((\mu, \lambda, \nu)\in M^*\times L^*\times Q^*\) the risk function (pay-off) is defined by \[ W= W(\mu, \lambda, \nu)= \varlimsup_{T\to \infty} {1\over T} \sum^T_{t= 1} P_{\mu\lambda\nu} (\xi_t= 1), \] where \(\xi_t\) is the state of the object under control at time \(t\) and for any cylindrical set \(J= J(u_0 x_1 z_1\cdots u_{t- 1} x_t z_t)\) the probability corresponding to the triple \((\mu, \lambda, \nu)\) satisfies \[ P_{\mu\lambda\nu}(J)= \prod^t_{s= 1} \nu_{s- 1} (u_{s- 1}|z^{s-1}, u^{s-2}) \mu(x_s|u_{s- 1}) \lambda_s(z_s|x^s, z^{s- 1}, u^{s- 1}). \] The paper introduces the concept of a normal observation process, and shows that if an observation process belongs to some normal class, then there exist a strategy which is adaptive in the entire class of partially observable processes with independent values. A method of adaptive strategy synthesis is described.