Subsampling estimates of the Lasso distribution.

5.1 Pointwise consistency for distribution estimation 45
For arbitrary ε > 0, set
E n (ε) =
{
τ b |θ(P ) − ˆθ
}
n | ≤ ε .
One then verifies that
U n (x − ε) ≤ 1 {E n (ε)} ≤ ˆL n,b (x)1{E n (ε)} ≤ U n (x + ε)
holds. Further, note that the assumption τ b /τ b → 0 implies P (E n (ε)) → 0, so for fixed ε,
with probaility tending to one, we obtain
For U n (x ± ε) → P J(x ± ε), we obtain
U n (x − ε) ≤ ˆL n,b (x) ≤ U n (x + ε)
J(x − ε, P ) ≤ ˆL n,b (x) ≤ J(x + ε, P ) + ε
with probability tending to one. Hence, taking a sequence ε n → 0, such that x ± ε n are
continuity points of J(·, P ) yields ˆL n,b (x) → P J(x, P ). Thus, it is sufficient to show that
U n (x) → P J(x, P ) for every continuity point x of J(·, P ).
For every 1 ≤ i ≤ N n , ˆθ n,b,i is a statistic based on a sample of size b drawn from the
distribution P , hence U n (x) is a U-statistic of degree b with
0 ≤ U n (x) ≤ 1 and E (U n (x)) = J b (x, P )
By Hoeffding’s ineqality ((Serfling, 1980, Theorem A, Section 5.6)), it follows that
P (U n (x) − J b (x, P ) ≥ t) ≤ exp
(−2⌊n/b⌋t 2)
for every t > 0. A similar inequality is obtained for t < 0 by considering −U n (x). So, it
follows that U n (x) → P J b (x, P ), for continuity points x of J(·, P ), this yields U n (x) → P
J(x, P ) since for such x, J b (x, P ) → J(x, P ) by the portmanteau theorem.
To prove ii, we use the subsequence criterion. Following i, given an arbitrary subsequence
{j n } n , for every continuity point x of J(·, P ), one can extract a further subsequence {k jn } n
such that L kjn (x) → J(x, P ) almost surely. By a diagonal argument, one can assume that
L kjn (x) → J(x, P ) almost surely for every x in a countable subset of the real line. So, we
obtain L kjn J(·, P ). By continuity of J(·, P ), it follows from Polya’s theorem that
∣
sup ∣L kjn (x) − J(x, P ) ∣ → 0
x∈R
almost surely, which completes the argument.
To prove iii, for α ∈ (0, 1), define
and
c L (1 − α) = inf {x ∈ R|J(x, P ) ≥ 1 − α}
c U (1 − α) = sup {x ∈ R|J(x, P ) ≤ 1 − α}
Then choose ε > 0 such that c L (1 − α) − ε and c U (1 − α) + ε are both continuity points
of J(·, P ). Following i, we have both
ˆL n,b (c L (1 − α) − ε) → P J(c L (1 − α) − ε, P )