Subsampling estimates of the Lasso distribution.

50 Subsampling
where the left hand side is the sum of n! identically distributed random ) variables, indeed
given some π ∈ S n , for every 1 ≤ i ≤ k n ,
(X π(b(i−1)+1) , . . . , X π(bi) is a sample of size b
from the distribution P . For an arbitrary ε > 0 we have
(
)
)
P
sup |Z n (x, P ; X 1 , . . . , X n )| > ε
x∈R
(
1
≤ P
n! sup |S n (x, P ; X π(1) , . . . , X π(n) )| > ε
x∈R
)
≤ 1 (sup
ε E |S n (x, P ; X 1 , . . . , X n )|
≤ 1 ε
x∈R
∫ ( 1
0
P
sup |S n (x, P ; X 1 , . . . , X n ) > u|
x∈R
where Markov’s inequality have been used in the second inequality and Fubini’s theorem
in the third one. Applying the Dvoretzky-Kiefer-Wolfowitz inequality 5.2.2.1 to the root
R b (X n,(b),i , P ), we obtain
P
(
sup |Z n (x, P ; X 1 , . . . , X n )| > ε
x∈R
)
≤ 1 ε
≤ 2 ε
∫ 1
0
2 exp(−2k n u 2 )du
√
2π
k n
(
Φ(2 √ k n − 1 2 ) )
≤ 1 ε
√
2πkn
where Φ(·) is the standard normal probability distribution function, this proves the first
inequality of the claim. Further, for arbitrary 0 < δ < 1, following previous arguments,
and after partitioning the unit interval according to δ, we obtain
(
)
≤ 1 (
)
ε E
P
sup |Z n (x, P ; X 1 , . . . , X n )| > ε
x∈R
sup |S n (x, P ; X 1 , . . . , X n )|
x∈R
≤ δ ε + 1 (sup
ε P |S n (x, P ; X 1 , . . . , X n )| > δ
x∈R
Applying the Dvoretzky-Kiefer-Wolfowitz inequality 5.2.2.1 to bound the second term on
the right hand side yields the second part of the claim.
Lemma 5.2.2.4. Let X (n) = (X 1 , . . . , X n ) be an i.i.d. sequence of random variables with
distribution P . Denote by J n (x, P ) the distribution of a real-valued root R n = R n (X (n) , P )
under P . Let k n = ⌊n/b⌋ and define L n (x, P ) according to 5.2.2.5. Then, for every ε > 0
and every 0 < γ < 1, the following hold:
(i) P (sup x∈R {L n (x, P ) − J n (x, P )} > ε) ≤
√ {
}
1 2π
+ 1 sup{J b (x, P ) − J n (x, P )} > (1 − γ)ε
γε k n x∈R
(ii) P (sup x∈R {J n (x, P ) − L n (x, P )} > ε) ≤
√ {
}
1 2π
+ 1 sup{J n (x, P ) − J b (x, P )} > (1 − γ)ε
γε k n x∈R
)
du
)