Strong consistency of Bayes estimates in stochastic regression models (Q1914696)

Consider the multiple regression model \[ y_n= \beta_1 x_{n1}+ \cdots+ \beta_p x_{np}+ \varepsilon_n, \qquad n=1, 2, \dots, \tag{1} \] where the \(\{\varepsilon_n \}^\infty_{n=1}\) are unobservable random errors, \(\beta^t= (\beta_1, \dots, \beta_p)\) are unknown parameters and \(y_n\) is the observed response corresponding to the design levels \(x_{n1}, \dots, x_{np}\). Let \({\mathbf x}_n^t= (x_{n1}, \dots, x_{np})\) and \({\mathbf X}_n= (x_{ij} )_{1\leq i\leq n, 1\leq j\leq p}\), \({\mathbf y}_n^t= (y_1, \dots, y_n)\). Then \[ {\mathbf b}_n^t= (b_{n1}, \dots, b_{np})= ({\mathbf X}^t_n {\mathbf X}_n )^{-1} {\mathbf X}^t_n {\mathbf y}_n \tag{2} \] denotes the least squares estimate (LSE) of \(\beta^t= (\beta_1, \dots, \beta_p)\), assuming \({\mathbf X}^t_n {\mathbf X}_n\) is nonsingular. (\(M^t\) denotes the transpose of a matrix \(M\).) The main result of this paper concerns stochastic design levels. Specifically, we assume that the design vector \({\mathbf x}_n\) at stage \(n\) depends on the previous responses and design levels \({\mathbf x}_1, y_1, \dots, {\mathbf x}_{n-1}, y_{n-1}\); i.e. \({\mathbf x}_n\) is \({\mathcal F}_{n-1}\) measurable, where \({\mathcal F}_{n-1}\) denotes the \(\sigma\)-field generated by \({\mathbf X}_{n-1}\) and \({\mathbf y}_{n-1}\). Without loss of generality we assume that \({\mathbf x}_1\) is a constant, and we shall take \({\mathcal F}_0\) to be the trivial \(\sigma\)-field \(\{\varnothing, \Omega\}\). This model is quite general and has been studied extensively in several fields of statistical inquiry. Time series models, dynamic input-output systems, stochastic approximation schemes, stochastic control, and sequential designs serve as primary examples. The strong consistency of estimates for \(\beta\) in model (1) is important to some applications, in particular, on-line identification and stochastic adaptive control of dynamic systems, and sequential design. Thus there has been considerable interest in the question of strong consistency of LSE in model (1).