Prediction and estimation consistency of sparse multi-class penalized optimal scoring (Q2278663)

The results on the consistency of group-Lasso estimators in linear regression cannot be applied to the penalized optimal scoring problem. The key difficulty is that linear model for the multiple response \(Y\) does not hold, and the dependency between the random covariates in \(X\) and the residual terms in \(E=Y-XB.\) The paper under review overcomes these challenges by using sub-exponential concentration bounds, and exploiting the decomposition of marginal covariance matrix. Main contributions are as follows. 1. A coordinate-sparse multi-class optimal scoring problem is formulated as the penalized multiple response linear regression problem. 2. The concentration bound is derived for the maximal row \(l_2\) norm of \(n^{-1}X^\top E\). Here \(X\) and \(E\) could be non-Gaussian and/or correlated. 3. Out-of-sample prediction and estimation bounds are derived for sparse multi-class optimal scoring problem which allow both the number of features and the number of classes to grow with the sample size. Bounds of two types are obtained: (a) slow-rate bounds which make no assumptions on the correlation structure of \(X\) or the sparsity of \(B\), and (b) fast-rate bounds which lead to faster convergence rates, but rely on exact sparsity of \(B\) and restricted eigenvalue condition. 4. It is shown that out-of-sample prediction consistency implies classification consistency, and finite-sample bounds are derived on misclassification error rate of multi-class penalized optimal scoring problem.