Subsampling estimates of the Lasso distribution.

35
exists a constant K d depending on d only such that
∥ ⎡ (∣ ∥∥∥∥ψd ∣∣∣∣ ∑
a i ε i ≤ K d
⎣E
n [
∑ n
n∑
∥
i=1
= K d
⎡
⎣E
≤ K d
⎡
⎢
≤ K d
⎡
⎣E
∣) ∣∣∣∣
a i ε i +
i=1
(∣( ∣∣∣∣ ∑ n
)
a i ε i · 1
∣
i=1
⎛( ∑ n
⎣E ⎝
(( n ∑
) ⎞ 2
a i ε i ⎠
i=1
a 2 i ε 2 i
i=1
] ⎤ 1/d ′
‖a i ε i ‖ d′ ⎦
ψ d
i=1
) [
∣ ∑ n
1/2
+
] ⎤ 1/d ′
|a i | d′ ‖ε i ‖ d′ ⎦
ψ d
i=1
[ n
] ⎤
+ (1 + K) 1/d C −1/d ∑ 1/d ′
|a i | d′ ⎥ ⎦
i=1
)) 1/2
+ (1 + K) 1/d C −1/d [ n ∑
⎡
[ n
] ⎤
=≤ K d
⎣σ + (1 + K) 1/d C −1/d ∑ 1/d ′
|a i | d′ ⎦
i=1
] ⎤ 1/d ′
|a i | d′ ⎦
i=1
Here, Hölder’s inequality and Lemma 4.0.2.4 have been used in the second inequality. For
1 < d ≤ 2, d ′ = d/(d − 1) ≥ 2, which implies
It follows that
(
n∑
n
)
∑ d ′ /2
|a i | d′ ≤ |a i | 2 = 1
i=1
i=1
∥ n∑ ∥∥∥∥ψd (
a
∥ i ε i ≤ K d σ + (1 + K) 1/d C −1/d)
i=1
For d = 1, by Lemma 4.0.2.7, there exists some constant K 1 such that
∥ (∣ n∑ ∥∥∥∥ψ1 ∣∣∣∣ a
∥ i ε i ≤ K 1
[E
n ∣) ]
∑ ∣∣∣∣
a i ε i + ‖ max |a iε i |‖ ψ1
1≤i≤n
i=1
i=1
]
≤ K 1
[σ + K ′ log(n) max ‖a iε i ‖ ψ1
1≤i≤n
]
≤ K 1
[σ + K ′ (1 + K)C −1 log(n) max |a i|
1≤i≤n
]
≤ K 1
[σ + K ′ (1 + K)C −1 log(n)
where Hölder’s inequality and Lemma 4.0.2.6 have been used in the second inequality.
Finally, note that an arbitrary random variable X, the following holds
P (X > t‖X‖ ψd ) ≤ (1 + ψ d (t)) −1 (1 + E (ψ d (|X|/‖X‖ ψd )))
≤ 2 exp(−t d )
for all t > 0, by Markov’s inequality and by definition of the ψ d -Orlicz norm.
(
Lemma 4.0.2.8. Let ˜s n1 = | ˜β
′
nj | −1 sgn(β 0j ))
and s n1 = ( | η˜
nj | −1 sgn(β 0j ) ) ′
j∈J j∈J n1
.
n1
Suppose that assumption B.2 holds. Then,
‖˜s n1 ‖ = (1 + o P (1)) M n1