Sorted concave penalized regression (Q2284364)

The linear model is considered \(~y=X\beta^*+\varepsilon,\) where \(X\in\mathbb{R}^{n\times p}\) is a design matrix, \(y\) is a response vector, \(\varepsilon\) is a noise vector and \(\beta^*\) is an unknown coefficient vector. It is assumed that \(\varepsilon\sim N(0,~\sigma^2I)\) and Euclidean norm of each column of \(X\) is equal to \(\sqrt n\). The paper focuses on local and approximate solutions for the minimization of penalized loss functions of the form \[ ||y-X\beta||^2_2/(2n)+\mathrm{Pen}(\beta) \] with a penalty function \(\mathrm{Pen}\) satisfying certain minimum penalty level and maximum concavity conditions. The sorted concave penalized estimation is proposed to combine the advantages of concave and sorted penalizations. The sorted concave penalties (SCP) adaptively choose the smaller penalty level and at the same time benefit from signal strength. Next, a local convex approximation for SCP is developed, which is easier to compute and possesses desired prediction and estimation error bounds. The analysis of prediction and estimation errors requires only the restricted eigenvalue condition on the design and provides selection consistency under a required minimum signal strength condition in addition. The results of the paper sharpen existing results on concave penalized least squares estimation by removing the upper sparse eigenvalue component of the sparse Riesz condition.