Subsampling estimates of the Lasso distribution.

38 The adaptive Lasso in a high dimensional setting
Now, let H n = I n − 1 n X n1C −1
n1 X′ n1 be the projection matrix on the nullspace of X′ 1 . By
definition of ˆβ n1 , we have
(
y − X n1 ˆβn1 = ε + X 1 β 01 − ˆβ
)
n1
= ε + 1 n X n1C −1 (
n1 X
′ )
1 ε − λ n˜s n1
= H n ε + λ n
n X n1C −1
n1 ˜s n1
Following condition 4.1.0.6 we have that ˆβ n = s β 0 if
⎧
⎨ sgn (β 0j )
(β 0j − ˆβ
)
nj < |β 0j | 1 ≤ j ≤ k n
(
)∣
⎩ ∣
∣x ′ j H n ε + λn n X n1C −1
n1 ˜s ∣∣ (4.1.0.7)
n1 < λn w nj , k n < j ≤ p n
Next, denote by e j , the j-th standard unit vector and choose an arbitrary 0 < κ < κ + δ <
1. One verifies that according to condition 4.1.0.7, ˆβn ≠ s β 0 implies the realization of
either of the following events :
B n1 =
B n2 =
B n3 =
B n4 =
k n ⋃
j=1
k n ⋃
j=1
⋃p n
j=k n+1
⋃p n
j=k n+1
{
}
n −1 |e ′ jC −1
n1 X 1ε| ≥ |β 0j |/2
{
}
|e j C −1
n1 ˜s n1| ≥ |β 0j |/2
{∣ ∣∣x ′
j H n ε∣
∣ ≥ (1 − κ − δ)λ n w nj
}
{
n −1 ∣ ∣ ∣x
′
j X 1 C −1
n1 ˜s ∣
n1
}
∣ ≥ (κ + δ) .
We show that they each have probability tending to zero as n → ∞. For B n1 , first note
that the matrix C n1 has root n 1/2 X n1 (X ′ n1 X n1) −1 . Thus we obtain
∥ ∥∥∥ ( ) ∥
n −1 e ′ jCn1 −1 ′ ∥∥∥ X′ 1 = n −1/2 ‖n −1/2 X n1 C −1
n1 e j‖
≤ n −1/2 ‖C −1/2
n1 ‖ ‖e j ‖
≤ (nτ n1 ) −1/2
by the spectral decomposition theorem. It follow that
⎛
⋃k n {
} ⎞
P (B n1 ) = P ⎝ ‖e ′ jC −1
n1 X′ 1ε‖/n ≥ |β 0j |/2 ⎠
≤ k n q ∗ n
1=1
( √ ) bn1 τn1 n
2
with q n (·) given in Lemma 4.0.2.7. Now, assumptions B.1, B.4 and B.5 imply that
P (B n1 ) → 0.
For the set B n2 , note that Lemma 4.0.2.6, then assumptions B.4 and B.5 imply that
λ n ∣
∣e j C −1
n1
n ˜s n1∣ ≤ λ ( )
n‖˜s n1 ‖ λn M n1
= O P = o P (b n1 ).
nτ n1
nτ n1