Subsampling estimates of the Lasso distribution.

Chapter 7
Concluding remarks
We reviewed the general theory of weak convergence with a focus on minizers of convex
processes and applied results together with tools from convex analysis to give a fairly
precise characterization of the limiting distribution of Lasso components in a low dimensional
setting, following the steps of Knight and Fu (2000). It was outlined that despite a
discontinuity at the point zero in the limiting distributions, the use of subsampling is still
justified to construct confidence intervals if the finite population distributions converge to
the limit uniformly, as pointed out recently in Romano and Shaikh (2010). We verified
this uniform convergence for an orthogonal design setting only but there are indications
that this property holds in greater generality; this remain an open problem.
In a high dimensional setting, where the use of the Lasso is most justified, it is more difficult
to study the large sample behavior in distribution. A major hurdle is that an eventual
limit to the objective function would not be uniquely minimized, thus prohibiting a direct
application of standard techniques from weak convergence theory. This explains why in
this setting, the asymptotics of the Lasso have not been adressed in the litterature in
a satisfactory way yet. Nevertheless, we investigated the adaptive Lasso under sparsity
assumptions in the vein of Huang et al. (2008) since this variant guarantees asymptotic
normality with optimal rate of convergence for estimates corresponding to nonzero coefficients
and it achieves variable selection, thus predicting assymptotic valid subsampling
confidence intervals for non-zero coefficients at least.
Two pictures emerge from the conducted simulation study. In a low dimensional setting,
the validity of subsampling was confirmed. Confidence intervals offer sastifying coverage
rates and proposed subsampled p-values allow to control the FWER through a Bonferonni-
Holm procedure. In a high dimensional setting however, coverage rates for nonzero coefficients
are slightly below the nominal level, this may be due to the bias introduced by the
Lasso. The conclusions for zero cofeffients are more interresting though; while convergence
to the nominal level was not achieved, probably due to variable selection consistency or
to a slower rate of convergence of order √ n/log(p n ), the rate of false positives is clearly
conservative, i.e. remains below the nominal level. Subsampled p-values however, seem
not to be valid in the high dimensional setting. This issue remains unsolved.
71