Universally consistent estimation for stochastic regression models (Q1902246)

Consider a multivariate linear regression model \[ y= {\mathbf b}^\tau {\mathbf x}+ \varepsilon.\tag{1} \] For the samples \((y_1, {\mathbf x}_1^\tau), \dots, (y_n,{\mathbf x}_n^\tau)\) from \((y, {\mathbf x}^\tau)\), the analogous model is \[ y_t= {\mathbf b}^\tau {\mathbf x}_t+ \varepsilon_t, \qquad t=1, 2, \dots, n. \tag{2} \] Estimating the unknown parameter \({\mathbf b}\) based on the sample \((y_t, {\mathbf x}_t^\tau)\), \(g=1, \dots, n\), is a crucial problem. The least squares estimator of \({\mathbf b}\) is popular because of its computational simplicity and mathematical beauty, especially in the Gaussian case. The \(M\)-estimator of \({\mathbf b}\) has been introduced more recently because of its robust properties. However, to ensure strong consistency of these estimators, some moment conditions on the distributions of \({\mathbf x}_t\) and \(\varepsilon_t\) must be imposed. However, in some practical problems, it is difficult to verify any such assumption on the distributions of \({\mathbf x}_t\) and \(\varepsilon_t\). Hence, we seek a new procedure to obtain a strongly consistent estimator of \({\mathbf b}\) under mild assumptions on the distributions of \({\mathbf x}_t\) and \(\varepsilon_t\). For this purpose, we make the following assumptions: (i) The series \(\{(y_t, {\mathbf x}_t^\tau)\}\) is a stationary and ergodic sequence with the same distribution as \((y, {\mathbf x}^\tau)\) which satisfies (1); (ii) \({\mathbf x}\) is not degenerate, i.e. there is no nonzero vector \({\mathbf c}\) such that \({\mathbf c}^\tau {\mathbf x}\) is a degenerate random variable; (iii) The series \(\{\varepsilon_t\}\) is i.i.d., and \(\varepsilon_t\) is independent of \(\{x_s: s\leq t\}\); hence, \(\varepsilon\) is independent of \({\mathbf x}\) in model (1).