A Predictive Least-Squares Principle

From MaRDI portal





This paper describes a new principle of least-squares estimation, called a predictive least-squares principle (PLSP). The basic idea behind PLSP is very simple: select the model (from the predetermined class) which minimizes the sum of squared prediction errors, when the observed data are predicted one-by-one using knowledge of the so-far processed observations only. This idea is best illustrated by considering the case of a linear regression model: \[ (1)\quad y(t)=\theta_ 1x_ 1(t)+...+\theta_ kx_ k(t)+e(t),\quad t=1,...,N \] where y(t) is the regressed variable, \(\{x_ k(t)\}\) are the regressors, \(\{\theta_ i\}\) are the unknown parameters and e(t) is a disturbance. Let \({\hat \theta}\)(s,k) denote the least-squares estimate of the parameters of (1), based on the first s samples \((t=1,...,s):\) \[ (2)\quad {\hat \theta}(s,k)=[\sum^{s}_{t=1}\phi_ k(t)\phi^ T_ k(t)]^{- 1}[\sum^{s}_{t=1}\phi_ k(t)y(t)] \] where \(\phi_ k(t)=[x_ 1(t)...x_ k(t)]^ T\). Then the PLS estimate of the model dimension k is obtained by minimizing the following criterion: \[ (3)\quad \sum^{N}_{s=1}[y(s)-{\hat \theta}(s-1,k)\phi^ T_ k(s)]^ 2. \] The main contribution of the paper is a proof of the fact that the value of k which minimizes (3) is a consistent estimate of the true model dimension, under the assumption that \(\{\) e(t)\(\}\) in (1) is a sequence of independent and identically distributed Gaussian random variables.











This page was built for publication: A Predictive Least-Squares Principle

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3030695)