De-Biasing The Lasso With Degrees-of-Freedom Adjustment

DOI10.3150/21-BEJ1348arXiv1902.08885MaRDI QIDQ6314520FDOQ6314520

Publication date: 23 February 2019

Abstract: This paper studies schemes to de-bias the Lasso in a linear model

where the goal is to construct confidence intervals for

in a direction

a_{0}

, where

X

has iid

N (0, S i g m a)

rows. We show that previously analyzed propositions to de-bias the Lasso require a modification in order to enjoy efficiency in a full range of sparsity. This modification takes the form of a degrees-of-freedom adjustment that accounts for the dimension of the model selected by Lasso. Let

s_{0}

be the true sparsity. If

S i g m a

is known and the ideal score vector proportional to

X S i g m a^{- 1} a_{0}

is used, the unadjusted de-biasing schemes proposed previously enjoy efficiency if

s_{0} l l l n^{2 / 3}

. However, if

s_{0} g g g n^{2 / 3}

, the unadjusted schemes cannot be efficient in certain

a_{0}

: then it is necessary to modify existing procedures by a degrees-of-freedom adjustment. This modification grants asymptotic efficiency for any

a_{0}

when

s_{0} / p o 0

and

s_{0} l o g (p / s_{0}) / n o 0

. If

S i g m a

is unknown, efficiency is granted for general

a_{0}

when

frac{s_0log p}{n}+minBig{frac{s_Omegalog p}{n},frac{|Sigma^{-1}a_0|_1sqrt{log p}}{|Sigma^{-1/2}a_0|_2 sqrt n}Big}+frac{min(s_Omega,s_0)log p}{sqrt n} o0

where

s_{O} m e g a = | S i g m a^{- 1} a_{0} |_{0}

, provided that the de-biased estimate is modified with the degrees-of-freedom adjustment. The dependence in

s_{0}, s_{O} m e g a

and

| S i g m a^{- 1} a_{0} |_{1}

is optimal. Our estimated score vector provides a novel methodology to handle dense

a_{0}

. Our analysis shows that the degrees-of-freedom adjustment is not needed when the initial bias in direction

a_{0}

is small, which is granted under stringent conditions on

S i g m a^{- 1}

. The main proof argument is an interpolation path similar to that typically used to derive Slepian's lemma. It yields a new

e l l_{i} n f t y

error bound for the Lasso which is of independent interest.

Mathematics Subject Classification ID

Statistics (62-XX) Information and communication theory, circuits (94-XX)

This page was built for publication: De-Biasing The Lasso With Degrees-of-Freedom Adjustment

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6314520)