Subsampling estimates of the Lasso distribution.

26 Application to the Lasso estimator
Conversely, suppose that 3.2.1.2 holds, we show that the minimizer necessarily satisfies
û 2 = 0. Indeed, if û 2 ≠ 0 then the equality is attained in 3.2.1.2 at every line corresponding
to nonzero components of û 2 , that is,
⎧
(
(
) ) ⎪⎨
−W 2 + C 21 C −1
11 (W 1 − λ 0 sgn(β)) + C 22 − C 21 C −1
11 C 12 û 2 = − λ 0
2 , if ûj 2 > 0
j ⎪ ⎩
λ 0
2 , if ûj 2 < 0
In particular, it follows from 3.2.1.2 that
{
((
) )
C 22 − C 21 C −1
11 C 12 û 2 ∈ [−λ0 , 0], if û j 2 > 0
j [0, λ 0 ] if û j 2 < 0 (3.2.1.3)
Note that (C 22 − C 21 C −1
11 C 12), the Schur complement of C, is SPD since C is. Let
D be the matrix which results from (C 22 − C 21 C −1
11 C 12) after removal of the lines and
columns corresponding to the zero components of û 2 , D is also SPD. Let û≠0 2 be the vector
which results from û 2 after removal of the zero components. Then 3.2.1.3 implies that
û≠0 T
2 Dû≠0 2 ≤ 0 which contradicts the fact that D is SPD, this complete the argument.
Finally, we show that every component of the limit distribution has a discontinuity at the
point zero only, and is otherwise Gaussian. Consider after an eventual permutation of the
last p − r covariates a further partitioning of W 2 , u 2 and C of the form
( )
W2,1
W 2 =
W 2,0
u 2 =
(
u2,1
u 2,0
)
,
and
C =
⎛
⎜
⎝
⎞
C 11 C 12 C 13
⎟
C 21 C 22 C 23 ⎠
C 31 C 32 C 33
respectively, where W 2,1 and u 2,1 are r ′ -vectors for some 0 < r ′ < p − r. For notational
convenience, denote
( )
C11 C ˜C = 12
.
C 21 C 22
Next, consider an arbitrary sign pattern s ′ ∈ {−1, +1} r′ . Then, given the event
{ sgn(u2,1 ) = s ′ ; u 2,0 = 0 } ,
(u 1 , u 2,1 ) ′ has the following distribution :
( ) (
( ))
u1
∼ N σ 2 λ 0 ˜C, −
u 2,1 2 ˜C −1 sgn(β)
s ′
This completes the proof.