Subsampling estimates of the Lasso distribution.

4.1 Variable selection consistency 37
For the second part of the claim, first note that
max ‖|η nj |˜s n1 − |η nj |s n1 ‖ 2 ∑k n ∣
= max ∣|η nj |/| ˜β ni | − |η nj |/|η ni | ∣ 2 ∑k n ≤ M 2 |η nj | − | ˜β ni |
n2
j>k n j>k n
∣
i=1
i=1
| ˜β nj ||η ni | ∣
(
) 2
≤ max || η nj | − | ˜β
k n 2
∑
nj ||
M n2
1≤j≤k n ∣|η ni |
(1 2 + | ˜β
)
ni |/|η ni | − 1 ∣
(
=
max || η nj | − | ˜β nj ||
1≤j≤k n
i=1
) 2 ∑k n ∣
i=1
= o P (1/r 2 n) O P (1) 2 M 2 n1 = o P (1).
M n2
|η ni | 2 (1 + o P (1)) ∣
Next, we have
∣ ∣∣| max ˜βnj | − |η nj | ∣ ‖˜s n1 ‖ = o P (1/r n )(1 + o P (1))M n1 = o P (1)
j>k n
The second part of the claim now follows from the triangle inequality.
2
2
4.1 Variable selection consistency
Theorem 4.1.0.9. (Huang et al., 2008, Theorem 1)(Variable selection consistency
of the adaptive Lasso) Suppose that assumptions B.1 to B.5 are valid, then
( )
P ˆβn = s β 0 → 1.
Proof. From subdifferential calculus, we know that a vector ˆβ n ∈ R pn is a solution to the
adaptive Lasso problem if the the subdifferential ∂L n ( ˆβ n ) of the objective function at ˆβ n
contains the point zero, that is
{ x
′ j (y − X ˆβ n ) = λ n w nj sgn( ˆβ nj ), if ˆβ nj ≠ 0
∣
∣x ′ j (y − X) ˆβ
∣ ∣∣
n < λn w nj , if ˆβ (4.1.0.5)
nj = 0
Furthermore, if the vector family
solution. Define
˜s n1 =
{
x j , ˆβ
}
nj ≠ 0 is linearly independent, then ˆβ n is unique
(
| ˜β n1 | −1 sgn(β 01 ), . . . , | ˜β
) ′
nkn | −1 sgn(β 0kn )
and
ˆβ n1 = ( X ′ n1X n1
) −1 ( X
′
n1 y − λ n˜s n1
)
= β 01 + 1 (
n C−1 n1 X
′ )
1 ε − λ n˜s n1
If ˆβ n1 = s β 01 then 4.1 holds for ˆβ
(
n = ˆβ′ n1, 0 ′) ′
. For this particular ˆβn we have X ˆβ n =
X 1 ˆβ n1 . The family {x j , 1 ≤ j ≤ k n } is then linearly independent, hence X ′ n1 X n1 regular
and we obtain that ˆβ n = s β 0 if
{
∣ ˆβn1 = s β 01
∣x ′ j (y − X n1 ˆβ (4.1.0.6)
n1 ) ∣ < λ n w nj , k n < j ≤ p n