Subsampling estimates of the Lasso distribution.

14 Minimizers of convex processes
Since every sequence {z (l)
j n n } n is Cauchy in l ∞ (T l ) by construction and
‖(z (l)
j n n
− z(l) jm ) |T l \T l−1
‖ |Tl \T l−1
≤ ‖z (l)
jn
− z(l) jm ‖ |T l
for l = 1, . . . , i, it follows that {(z j n n
) |Ti } n is Cauchy in l ∞ (T i ).
(ii) Using compactness of the sets K i one easily verifies that for arbitrary δ > 0, it holds
that
{
}
z ∈ l ∞ (T 1 , T 2 , . . . ) | z |Ti ∈ Ki
δ for every i ∈ N ⊆ K δ .
This implies that
(
lim sup P X n /∈ K δ) ≤
n→∞
∞∑
P
i=1
(
X n|Ti
)
/∈ Ki
δ < ε
for every δ > 0, that is, {X n } n
is asymptotically tight in l ∞ (T 1 , T 2 , . . . ).
Then, consider the set F of continuous functions f : l ∞ (T 1 , T 2 , . . . ) → R of the form
f(z) = g(z(t 1 ), . . . , z(t k )) with g ∈ C b (R), t 1 , . . . , t k ∈ T and k ∈ N. Obbviously
F is an algebra that separates points in l ∞ (T 1 , T 2 , . . . ). Lemma 2.2.1.9 now implies
that {X n } n is asymptotically measurable in l ∞ (t 1 , T 2 , . . . ).
(iii) Finally Prohorov’s theorem 2.2.1.12 asserts that every subsequence of {X n } n has a
further subsequence which converges weakly to a tight limit. This limit is unique in
viwe of Lemma 2.2.1.8
Corollary 2.2.2.6. (Weak convergence of convex processes) Let {X n } n be a sequence
of stochastic processes indexed by a convex, open subset C of R p such that every
sample path t ↦→ X n (ω, t) is convex on C. If {X n } n converges marginally in distribution
to a limit, then it converges in distribution to a tight limit in the space l ∞ (K 1 , K 2 , . . . , )
for any sequence of compacts sets K 1 ⊂ K 2 ⊂ · · · ⊂ C. In particular, it converges to a
tight limit with respect to the topology of uniform convergence on compacta.
Proof.
(i) We show that for every compact K ⊂ C, there exists an ε > 0 such that K ε ⊂ K 2ε ⊂
C. In particular, ‖f‖ K ε = sup x∈K ε |f(x)| is well defined for every convex function
f : C → R.
Indeed, for each x ∈ K one can find an ε x such that B(x, 2ε x ) ⊂ C. Let ∪ i∈I B(x i , 2ε xi )
be a finite covering of K with such balls. Set ε = min i∈I ε xi . One easily verifies that
K 2ε ⊂ ∪ i∈I B(x i , 4ε) then holds.
(ii) Next, we first show that a sequence {x n : C → R} n of deterministic convex functions
such that {x n (t)} n is bounded for every t is automatically uniformly bounded over
K ε , i.e. sup n ‖x n ‖ K ε < ∞
For such functions t ↦→ sup n |x n (t)| is finite and convex on C, hence uniformly
continuous on every compact subset of C. This implies that for arbitrary ε > 0
there exists some δ > 0 such that for every t 0 ∈ K ε , sup n |x n (·)| ≤ sup n |x n (t 0 )| +
ε on B(t 0 , δ). Cover K ε by finitely many balls B(t i , δ), i = 1, . . . , N. Then it holds
that sup n ‖x n ‖ K ε ≤ max i sup n |x n (t i )| + ε.
We can then conclude that the sequence {‖X n ‖ K ε} n is asynmptotically tight.