Subsampling estimates of the Lasso distribution.

16 Minimizers of convex processes
(ii) X n converges almost uniformly to X if for every ε > 0, there exists a measurable set
A with P (A) ≥ 1 − ε and d(X n , X) → 0 uniformly on A; this is denoted X n → au X.
Almost uniform convergence is more convenient to prove, so we will make use of
Lemma 2.3.0.9. (Van der Vaart and Wellner, 1996, Lemma 1.9.2)Let X be Borel measurable.
Then X n → au X if and only if X n → as∗ X.
Theorem 2.3.0.10. (Dudley’s almost sure representation theorem)
For probability spaces (Ω n , A n , P n ), n ∈ N ∪ {∞}, let X n : Ω n → D be arbitrary maps and
let X ∞ : Ω → D be Borel measurable and separable. If X n X ∞ , then there exists a
probability space (˜Ω, Ã, ˜P ) and maps X n : ˜Ω → D, n ∈ N ∪ {∞} with
(i) ˜X n → as∗ ˜X∞
(
(ii) E ∗ f( ˜X
)
n ) = E ∗ (f(X n )), for every bounded f : D → R and every n ∈ N ⋃ {∞}.
Moreover each ˜X n can be chosen as ˜X n = X n ◦ φ n with φ n measurable and perfect, and
P n = ˜P ◦ φ −1
n .
Proof. Call a set B ⊂ D a continuity set if P (X ∞ ∈ ∂B) = 0.
(i) We first show that for every ε > 0, there exists a partition of D into finitely many
disjoint continuity sets B (ε)
0 , B(ε) 1 , . . . , B(ε) k ε
satisfying
(
P X ∞ ∈ B (ε) )
0 < ε 2 (2.3.0.2)
and
diam(B (ε)
i ) < ε (2.3.0.3)
for i = 1, . . . , k ε . Let C ⊂ D be a separable subset for which P (X ∞ ∈ C) = 1 and let
{s i } i be a dense sequence in C. For each i ∈ N, there are at most countably many
values r > 0 for which the open ball B(s i , r) is a discontinuity set (every σ−finite
measure space has at most countably many disjoint sets with positive measure). So,
choose ε/3 < r i < ε/2 such that B(s i , r i ) is a continuity set. Then for every i, set
B (ε)
i = B(s i , r i )\ ⋃ j 1 − ε 2 , so set B (ε)
0 = D \ ⋃ i≤k ε
B (ε)
i .
(ii) There is a sequence ε n → 0 taking values 1/m (m ∈ N) only, such that
(
P n∗ X n ∈ B (εn) )
(
≥ (1 − ε n ) P ∞ X ∞ ∈ B (εn) )
, i = 1, . . . , k εn . (2.3.0.4)
i
By the ( Portmanteau theorem, for fixed ε > 0 and i = 1, . . . , k ε , it holds that
P n∗ X n ∈ B (εn) ) (
i → P ∞ X ∞ ∈ B (εn) )
i . So, for m ∈ N choose n(m) be such that for
i = 1, . . . , k 1/m and n ≥ n(m)
(
P n∗ X n ∈ B (1/m) )
(
≥ (1 − 1/m) P ∞ X ∞ ∈ B (1/m) )
, (2.3.0.5)
i
i
i