The reproducing kernel Hilbert space approach in nonparametric regression problems with correlated observations (Q2027227)

The paper investigates the estimation of a regression function \(g\) in a longitudinal study when measurement errors are correlated. More precisely, the available data is made of the \(Y_j(t_i)\), where \[ Y_j(t_i)=g(t_i)+\varepsilon_j(t_i), \quad 1\leq i\leq n, \ 1\leq j\leq m, \] and the \(\varepsilon_j\) are i.i.d.~unobserved stochastic processes. The observation points \(t_i\) are nonrandom, thus putting the model into the so-called fixed design case. The autocovariance function \(R\) of the \(\varepsilon_j\) is assumed to be known, which allows the construction of a projection estimator in the associated reproducing kernel Hilbert space RKHS(\(R\)), see Definition~1 in the paper. In general, this estimator does not have a simple closed form, but an expression is provided when the error process is a Wiener process or an Ornstein-Uhlenbeck process. The bias, variance, and integrated mean squared error of the estimator are considered in Section~3, and a comparison with the estimator of \textit{T. Gasser} and \textit{H.-G. Müller} [Lect. Notes Math. 757, 23--68 (1979; Zbl 0418.62033)] is provided in Section~4. It is in particular shown that the Gasser-Müller estimator has a larger variance although, contrary to the proposed estimator, it does not require the knowledge of the autocovariance structure of the error function. A simulation study is provided in Section~5 to ascertain the finite-sample performance of the proposed method.