One-step estimator paths for concave regularization

DOI10.48550/ARXIV.1308.5623arXiv1308.5623MaRDI QIDQ87161FDOQ87161

Publication date: 26 August 2013

Abstract: The statistics literature of the past 15 years has established many favorable properties for sparse diminishing-bias regularization: techniques which can roughly be understood as providing estimation under penalty functions spanning the range of concavity between

L_{0}

and

L_{1}

norms. However, lasso

L_{1}

-regularized estimation remains the standard tool for industrial `Big Data' applications because of its minimal computational cost and the presence of easy-to-apply rules for penalty selection. In response, this article proposes a simple new algorithm framework that requires no more computation than a lasso path: the path of one-step estimators (POSE) does

L_{1}

penalized regression estimation on a grid of decreasing penalties, but adapts coefficient-specific weights to decrease as a function of the coefficient estimated in the previous path step. This provides sparse diminishing-bias regularization at no extra cost over the fastest lasso algorithms. Moreover, our `gamma lasso' implementation of POSE is accompanied by a reliable heuristic for the fit degrees of freedom, so that standard information criteria can be applied in penalty selection. We also provide novel results on the distance between weighted-

L_{1}

and

L_{0}

penalized predictors; this allows us to build intuition about POSE and other diminishing-bias regularization schemes. The methods and results are illustrated in extensive simulations and in application of logistic regression to evaluating the performance of hockey players.

Cited In (1)

gamlr

This page was built for publication: One-step estimator paths for concave regularization

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q87161)