Subsampling estimates of the Lasso distribution.

24 Application to the Lasso estimator
a convex function minimized at √ ( )
n ˆβ n − β . We have
n∑
i=1
(
(ε 2 i − u ′ x i / √ )
n) 2 − ε 2 i =
n∑
( 1
n u′ x i x ′ iu − 2ε i u ′ x i / √ n)
i=1
= 1 ( n
)
∑
n∑
n u′ x i x ′ i u − −2ε i u ′ x i / √ n
i=1
i=1
Set ξ i = ε i x i / √ n, under the assumption 3.0.0.4, it holds that
n∑
i=1
(
)
E ‖ξ i ‖ 2 1 {‖ξi ‖>ε} =
=
≤
n∑
i=1
n∑
i=1
n∑
i=1
(
)
E ‖ξ i ‖ 2 1{‖ξ i ‖ > ε}
1 (
n ‖x i‖ 2 E |ε i | 2 1{|x i ‖|ε i |/ √ )
n > ε}
1
n ‖x i‖ 2 E
= tr(C n ) E
(
|ε i | 2 1{|ε i | max ‖x i‖/ √ )
n > ε}
1≤i≤n
(
|ε 1 | 2 1{|ε 1 | max
1≤i≤n ‖x i||/ √ n > ε}
This converges to zero by Lebesgue’s dominated convergence theorem since tr(C n ) = p.
The Lindeberg-Feller theorem 3.2.0.14 now implies that
n∑
i=1
(
(ε 2 i − u ′ x i / √ )
n) 2 − ε 2 i −2u ′ Wu + u ′ Cu
with W ∼ N (0, C) for every u ∈ R p .
On the other hand we have
p∑ ( |βj + u j / √ n| − |β j | ) p∑
→ λ 0 (sgn(β j )I(β j ≠ 0) + |u j |I(β j = 0)) .
λ n
j=1
j=1
Now weak convergence of the marginals of V n to the marginals of V , both viewed as random
maps into R p follows directly from the Cramer-Wold device. Since C is nonsingular the
process V is strictly convex, hence takes values in C min (R p ). In view of the Argmin
continuous mapping theorem 2.3.0.11, this completes the proof.
)
.
3.2.1 Limiting distribution of components
In this section, we further investigate the asymptotic behavior in distribution of single
components the root √ n( ˆβ n − β), our goal being the construction of confidence intervals
and hypothesis tests for individual coefficients.
In the next proposition, we show that, in the limit, the subvector of the root corresponding
to zero parameters can take value zero with positive probability, this probability is
quantified. Also, we show that the limiting distribution of each component has at most a
discontinuity at the point zero, that it is otherwise Gaussian.
For notational convenience, in the remaining, we assume without loss of generality that
β 1 , . . . , β r are nonzero and that β r+1 = · · · = β p = 0, this yields
⎛
⎞
r∑
p∑
V (u) = −2u ′ W + 2u ′ Cu + λ 0 ⎝ sgn(β j )u j + |u j | ⎠ .
j=1
j=r+1