Subsampling estimates of the Lasso distribution.

46 Subsampling
and
ˆL n,b (c U (1 − α) + ε) → P J(c U (1 − α) + ε, P ).
Hence, the sets
{ˆLn,b (c L (1 − α) − ε) < 1 − α ≤ ˆL n,b (c U (1 − α) + ε)}
⊆
{
}
−1
c L (1 − α) − ε < ˆL
n,b (1 − α) ≤ c U(1 − α) + ε
have probability tending to one as n → ∞. It follows that
(
)
P τ n (ˆθ n − θ(P )) ≤ ĉ n,b (1 − α) ≤ J n (c U (1 − α) + ε, P ) + o(1)
and
P
(
)
τ n (ˆθ n − θ(P )) ≤ ĉ n,b (1 − α) ≥ J n (c L (1 − α) − ε, P ) + o(1).
Letting n tend to infinity first, then ε tend to zero yields, together with the Portmanteau
Theorem, the inequalities.
Finally iv can be proved similarly to i and ii using Borel-Cantelli Lemma.
Remark.
(i) Note that point iii also holds for the root U n,b . Indeed, the proof for ˆL n,b (·) solely
rests on the convergence in probability of ˆL n,b (x) to J(x, P ) for every continuity
point x of J(·, P ). As seen in the proof of i, this is a property shared by U n,b (·) as
well, this without even requiring τ b /τ n → 0, the assumption b/n → 0 being sufficient.
Obviously the price to pay are larger confidence intervals.
(ii) The conclusion of point iii can also be stated for two-sided confidence intervals with
obvious changes in the assumptions.
In the regular situation where τ n = √ n, the choice b = n δ for some 0 < δ < 1 satisfies the
conditions of Theorem 5.1.0.12.
In view of our goal, constructing confidence intervals for Lasso estimates, the message
conveyed by Theorem 5.1.0.12 is that in the situation where the 1 − α quantile happens
to be a discontinuity point, and this can indeed happen if the corresponding parameter is
equal to zero (cf. Theorem 3.2.1.1), the subsampling confidence interval assymptotically
carries an error which is in the worst case equal to the jump height at the quantile.
However, as we will see in the next section, this conclusion is too pessimistic and it turns
out that some form of uniform convergence is what we need to achieve consistency.
5.2 Uniform consistency for quantiles appproximation
The present section focuses on the use of subsampling for the construction of confidence
intervals only, in contrast to the previous one where the estimation of the distribution
function in a uniform sense was also considered. We will see that achieving asymptotic
valid or conservative confidence intervals is possible if the distribution functions satisfy
some uniformity or monoticity condition in the limit.
All results of this section, appart the Dvoretzky-Kiefer-Wolfowitz inequality, are due to
Romano and Shaikh (2010) who stated their results in a uniform sense for a family of
probability measures, we follow their exposition.