Subsampling estimates of the Lasso distribution.

2.1 Convergence in probability 5
for every x ∈ C, where f is a real (random) variable on C. Further, assume that f is
uniquely minimized at α ∈ C and let {α n } n be a sequence of minimizers of {f n } n , i.e.
α n ∈ arg min
x∈C
f n (·), n ∈ N.
Then,
α n → P α.
Proof. The proof rests on (Hjort and Pollard, 1993, Lemma 2) and subsequent remarks.
For arbitrary δ > 0, define
and
∆ n (δ) =
h(δ) =
sup |f n (x) − f(x)|
‖x−α‖≤δ
inf f n(x) − f n (α)
‖x−α‖=δ
First, we show that
P (‖α n − α‖ ≥ δ) ≤ P
(∆ n (δ) ≥ 1 2 h(δ) )
. (2.1.0.2)
For every x ∈ R p with ‖x − α‖ > δ there exist some c with ‖c − α‖ = δ and t ∈ [0, 1] such
that c = tx + (1 − t)α. By convexity of the functions f n , we have
f n (c) ≤ tf n (x) + (1 − t)f n (α).
for every n ∈ N. Writing r n (x) = f(x) − f n (x), we obtain
t(f n (x) − f n (α)) ≥ f n (c) − f n (α)
= f(c) − f(α) + r n (c) − r n (α)
≥ h(δ) − 2∆ n (δ)
In particular,
{∆ n (δ) < 1 }
2 h(δ) ⊆ {f n (x) > f n (α)}
for every x with ‖x − α‖ > δ and every n ∈ N. Since α minimizes f n , this implies that
{‖α n − α‖} ⊆
{∆ n (δ) ≥ 1 }
2 h(δ)
and 2.1.0.2 follows.
Next, we show that
∆ n (δ) = o P (1) (2.1.0.3)