The linearized alternating direction method of multipliers for sparse group LAD model (Q2421443)

The linear regression problem \(Y=X\beta+\varepsilon,\) with \(X,\) a real valued \(n\times p\) matrix of covariates, \(Y,\) a real response \(n\) vector, \(\beta,\) the regression coefficient \(p\) vector and \(\varepsilon,\) the error \(n\) vector, is considered. In the high dimensional case \( (n < p),\) under the key assumption that sparsity of the true coefficient parameters holds, one usually uses, for estimating the unknown parameters, the least absolute shrinkage and selection operator (Lasso). Variations of this method can deal with the case where the regression coefficients form groups of variables. The Lasso-type methods have good properties when the error distribution is Gaussian or near Gaussian. When this does not hold (e.g., the errors are heavy-tailed or heterogeneous) one has the penalized least absolute deviation \((L_1-LAD)\) model providing more robust estimations than the Lasso model. This paper develops the sparse group LAD (SGLAD) model for dealing with sparsity at both group-wise and within group level, when robustness is desirable. Under conveniently chosen penalty level the paper shows that SGLAD \(L_2\) norm of the estimation error is \(O(\sqrt{\frac{k\log p}{n}}),\) where \(k\) is the number of nonzero true coefficient parameters. In order to find the estimations of the SGLAD model the linearized alternating direction method of multipliers algorithm is used and to prove that this last algorithm converges, the KKT condition of the SGLAD model is reformed as variational inequalities. Numerical experiments are exhibited to show the efficiency of the proposed method.