Subsampling estimates of the Lasso distribution.

22 Application to the Lasso estimator
for some c > 0.Then
almost surely.
1
n
n∑
ξ i → 0
i=1
Theorem 3.1.0.13. (Convergence in probability) If C is nonsingular and λ n /n →
λ 0 ≥ 0, then
where
ˆβ n → P arg min(Z)
p∑
Z(φ) = (φ − β) ′ C(φ − β) ′ + λ 0 |φ j |.
j=1
Thus if λ n = o(n), arg min(Z) = β and so ˆβ n is consistent.
Proof. Maps Z n defined in 3.0.0.1 have convex sample paths and are minimized at ˆβ n .
Sample paths of Z are strictly convex since C is nonsingular, hence have unique minimizers.
Following Corollary 2.1.0.3, it is sufficient to show that Z n (φ) → P Z(φ) + σ 2 for every
point φ ∈ R p .
We have
Set ξ i = ε i (β − φ) ′ x i with
Z n (φ) = 1 n∑
(Y i − x ′
n
iφ) 2 + λ n
p∑
|φ j |
n
i=1
j=1
= 1 n∑
(x ′
n
iβ + ε i − x ′ iφ) 2 + λ n
p∑
|φ j |
n
i=1
j=1
( )
1
n∑
= (β − φ) ′ x i x ′ i (β − φ) + 1 n∑
ε i
n
n
i=1
i=1
+ 2 n∑
ε i (β − φ) ′ x i + λ n
p∑
|φ j |
n
n
i=1
j=1
i=1
.
E(ξ 2 i ) = σ 2 |〈β − φ, x i 〉| 2
≤ σ 2 ‖(β − φ)‖ 2 ‖x i ‖ 2 .
Under assumption 3.0.0.4 i −1 ‖x i ‖ 2 is asymptotically bounded above by i −δ for some δ > 0,
thus we obtain
∞∑ 1
∞
i 2 E(ξ2 i ) ≤ σ 2 ‖(β − φ)‖ 2 ∑
( ) 1 2
i ‖x i‖ < ∞.
Now it follows from theorem 3.1.0.12 that
2
n
i=1
n∑
ε i (β − φ) ′ x i → 0
i=1