Asymptotic risk and phase transition of \(l_1\)-penalized robust estimator (Q2215774)

Consider a high-dimensional regression model where the number \(p\) of parameters exceeds the number \(n\) of observations. For estimation of the parameter vector often penalized estimators are used where the tuning parameter controls both variable selection and regularization. If least square loss is used this penalized estimation procedure is called LASSO (least absolute shrinkage and selection operator). The author investigates penalized robust estimators with least absolute deviation (LAD) loss and with Huber's loss. Especially he derives the asymptotic mean square error (MSE) if both \(n\) and \(p\) going to infinity but with fixed ratio \(\frac{n}{p}=\delta\). Let be \(k< p\) the number of nonzero regression parameters and \(\eta=\frac{k}{p}=\epsilon\) the sparsity rate. In the \((\epsilon,\delta)\)-plane there is a phase transition from bounded to unbounded asymptotic risk. The author shows that the phase transition boundary curve for LAD loss and Huber's loss estimation is identical to the curve of LASSO. The analysis is based on application of the generalized approximate message passing (GAMP) algorithm to penalized robust regression. All analytical results are confirmed by numerical experiments on finite-size systems. The author's formulas work well even for moderate-size systems.