Subsampling estimates of the Lasso distribution.

52 Subsampling
Proof. We prove (iii), (i) and (ii) are established by similar arguments. By part (iii) of
Lemma 5.2.2.4, for arbitrary ε > 0 and 0 < γ < 1, we have
(
)
P
sup |L n (x, P ) − J n (x, P )}| ≤ ε
x∈R
≥ 1 − δ 3,n (ε, γ, P )
The claim directly follows from the last claim of Lemma 5.2.2.2 with Ĝ(x) = L n(x, P ) and
X = R n (X (n) , P ) with probability distribution function F (x) = J n (x, P ).
We are now able to prove Theorem 5.2.1.1.
Proof. We prove (iii), the arguments for (i) and (ii) go along the same lines. For arbitrary
ε > 0 and 0 < γ < 1, define
δ 3,n (ε, γ, P ) = 1
√ {
}
2π
+ 1 sup{|J b (x, P ) − J n (x, P )|} > (1 − γ)ε
γε k n x∈R
By Lemma 5.2.2.5, we have
(
)
L −1
n (α) ≤ R n ≤ L −1
n (1 − α) ≥ 1 −
(
inf P
P ∈P
α + ε + sup{δ 3,n (ε, γ, P )}
P ∈P
Under the assumption lim n→∞ sup P ∈P sup x∈R |J n (x, P ) − J b (x, P )| = 0, for ε fixed we
have sup P ∈P {δ 3,n (ε, γ, P )} → 0. By a diagonal argument, one argues that there exists a
sequence ε n ↘ 0 such that sup P ∈P {δ 3,n (ε n , γ, P )} → 0. Applying Lemma 5.2.2.5 to the
sequence {ε n } n∈N yields
lim inf
n→∞
(
inf P
P ∈P
L −1
n
)
(α) ≤ R n ≤ L −1
n (1 − α) ≥ 1 − 2α
For the other inequality we show that lim sup n→∞ inf P ∈P P ( L −1
n (α) ≤ R n ≤ L −1
n (1 − α) ) ≤
1 − 2α. Consider an arbitrary P 0 ∈ P. Then for every ε > 0 and 0 < γ < 1,
(
)
P 0 L −1
n (α) ≤ R n L −1
n (1 − α)
(
)
= P 0 L −1
n (α) ≤ R n ≤ L −1
n (1 − α); sup |J n (x) − L n (x)| ≤ ε
x∈R
)
+ P 0
(
≤ P 0
(
L −1
n
L −1
n
(α) ≤ R n ≤ L −1
n
(α) ≤ R n ≤ L −1
n
+ P 0
(sup |J n (x) − L n (x)| > ε
x∈R
(
)
≤ P 0 Jn
−1 (α) ≤ R n ≤ Jn −1 (1 − α) + δ 3,n (ε, γ, P 0 )
= 1 − 2α + δ 3,n (ε, γ, P 0 )
(1 − α); sup |J n (x) − L n (x)| > ε
x∈R
)
(1 − α); Jn
−1 (α − ε) ≤ L −1
n (α); L −1
n (1 − α) ≤ Jn −1 (1 − α + ε)
)
where Lemma 5.2.2.3 have been used in the first inequality. Further, under the assumption
lim n→∞ sup P ∈P sup x∈R |J n (x, P ) − J b (x, P )| = 0, we have
This completes the proof.
1 − 2α + δ 3,n (ε, γ, P 0 ) → 1 − 2α.
)