De-Biasing The Lasso With Degrees-of-Freedom Adjustment

This paper studies schemes to de-bias the Lasso in a linear model $y=X\beta+\epsilon$ where the goal is to construct confidence intervals for $a0^T\beta$ in a direction $a0$, where $X$ has iid $N(0,\Sigma)$ rows. We show that previously analyzed propositions to de-bias the Lasso require a modification in order to enjoy efficiency in a full range of sparsity. This modification takes the form of a degrees-of-freedom adjustment that accounts for the dimension of the model selected by Lasso. Let $s0$ be the true sparsity. If $\Sigma$ is known and the ideal score vector proportional to $X\Sigma^{-1}a0$ is used, the unadjusted de-biasing schemes proposed previously enjoy efficiency if $s0\lll n^{2/3}$. However, if $s0\ggg n^{2/3}$, the unadjusted schemes cannot be efficient in certain $a0$: then it is necessary to modify existing procedures by a degrees-of-freedom adjustment. This modification grants asymptotic efficiency for any $a0$ when $s0/p\to 0$ and $s0\log(p/s0)/n \to 0$. If $\Sigma$ is unknown, efficiency is granted for general $a0$ when $$\frac{s0\log p}{n}+\min\Big{\frac{s\Omega\log p}{n},\frac{|\Sigma^{{-1}a0|1\sqrt{\log} p}}{|\Sigma^{-1/2}a0|2 \sqrt n}\Big}+\frac{\min(s\Omega,s0)\log p}{\sqrt n}\to0$$ where $s\Omega=|\Sigma^{-1}a0|0$, provided that the de-biased estimate is modified with the degrees-of-freedom adjustment. The dependence in $s0,s\Omega$ and $|\Sigma^{-1}a0|1$ is optimal. Our estimated score vector provides a novel methodology to handle dense $a0$. Our analysis shows that the degrees-of-freedom adjustment is not needed when the initial bias in direction $a0$ is small, which is granted under stringent conditions on $\Sigma^{-1}$. The main proof argument is an interpolation path similar to that typically used to derive Slepian's lemma. It yields a new $\ell\infty$ error bound for the Lasso which is of independent interest.