Dual‐ and triple‐mode matrix approximation and regression modelling (Q4827962)

Let \(X=\{x_{ij}\}\) denote a data matrix. A matrix approximation by two vectors is \(X=\lambda ba'+\epsilon\), where \(b\) is a column-vector of \(m\)-th order, \(a'\) is a row-vector of \(n\)-th order, \(\lambda\) is a constant parameter, \(\epsilon\) is a matrix of residuals, and \(a'a=1,\;b'b=1\). The least squares for singular value decomposition (SVD) determines \(a\) and \(b\) vectors by minimizing the objective \(LS(SVD)=\| X-\lambda ba'\|^{2}\). In the regression modelling \(y=Xa+\epsilon\), \(X\) is a matrix of independent variables, \(y\) is the vector of dependent variables, and \(\epsilon\) is a vector of deviations. The least squares for the regression analysis consists in minimizing the objective \(LS(RA)=(y-Xa)'(y-Xa)\), where the norm of the standardized vector \(y\) equals one. The authors consider a model of dual relations, \(b=Xa+\epsilon\), \(a=X'b+\delta\), for singular value decomposition (SVD). Both \(a\) and \(b\) vectors can be found via minimizing the conditional objective \[ DLS(SVD)=\| b-Xa\|^{2}+\| a-X'b\|^{2}-\lambda(a'a-1)-\mu(b'b-1). \] For the regression model we have the relations \(y=Xa+\epsilon\), \(a=X'y+\delta\). The objective in this model is the following: \(DLS(RA)=\| y-Xa\|^2+\| a-X'y\|^2\). The authors also consider a triple-mode least squares objective \[ TLS(SVD)=\| b-Xa\|^{2}+\| a-X'b\|^{2}+\| X-\lambda ba'\|^{2}-\lambda(a'a-1)-\mu(b'b-1) \] for \(SVD\), and the triple-mode least squares objective \[ TLS(RA)=\| y-Xa\|^2+\| a-X'y\|^2+\| X-ya'\|^{2} \] for regression modelling. When applied to \(SVD\) the proposed approach generates a regular solution with a new interpretation of the relations between the paired vectors. Applied to regression modelling, this approach produces regularized objectives and yields solutions not prone to multicollinearity effects. Several quantities for measuring the differences between ordinary and modified solutions are presented. Numerical results are given for a marketing research data set.