Inference without compatibility: using exponential weighting for inference on a parameter of a linear model (Q2040072)

The authors consider a partially linear model \(Y = X\beta + \mu + \epsilon\) with \(n\)-dimensional \(Y\) and \(q\)-dimensional \(\beta\). The parameter \(\beta\) is assumed to be low-dimensional, \(q<n\), but the random nuisance \(\mu\) is modeled by \(\mu = Z\gamma\) with a \(p\)-dimensional sparse vector \(\gamma\), where \(p\) is high-dimensional, \(p>n\). The aim of the authors is inference on \(\beta\), on the signal strength \(\sigma^2_\mu\) and on the noise variance \(\sigma^2_\epsilon\). Since the Lasso (least absolute shrinkage and selection operator) forms the basis for the procedure, certain assumptions must be made to ensure nice properties. Especially the so-called compatibility condition seems to be indispensible. The main contribution of this paper is proving that the compatibility condition is not necessary for the statistical problem. The authors are able to construct \(\sqrt n\)-consistent estimators for \(\beta\), \(\sigma^2_\mu\) and \(\sigma^2_\epsilon\). Their approach involves using exponential weighing to aggregate over all models of particular size. This is a computationally hard problem, but can be well approximated. To this end, the authors present an algorithm and simulation results which allow comparison with other estimators.