Subsampling estimates of the Lasso distribution.

Chapter 1
Introduction
In this thesis we consider the linear regression model
Y i = x ′ iβ + ε i , i = 1, . . . , n
and focus on the Least Absolute Shrinkage and Selection Operator, or Lasso, as estimation
method for the parameter β. The Lasso, defined as
arg min
φ∈R p
∑ n p∑
(x ′ iφ − Y i ) 2 + λ |φ| j ,
i=1
j=1
was introduced by Tibshirani (1996) and has become very popular over the last years.
There are two main reasons for this gain in popularity. The first being, in a context of
high dimensional data analysis, where the dimension of the parameter is very large, by introducing
an l 1 -norm penalty in addition to the squared loss, the Lasso can induce sparsity
on the estimated model while maintaining its prediction ability. That is, many coefficients
will be set to zero, which is a desirable property if one believes that only few parameters
are relevant. The second reason is that the solution can be computed efficiently by convex
optimization. Efron, Hastie, Johnstone, and Tibshirani (2004) introduced the LARS
algorithm which computes the entire solution path, that is, estimates corresponding to all
values λ > 0 with the cost of a single least square regression.
For the sole purpose of prediction, the Lasso proves to be very efficient. Several authors
showed under various assumptions that it enjoys a so-called oracle property, see for instance
Zou (2006), Bunea, Tsybakov, and Wegkamp (2007) or Van De Geer and Bühlmann
(2009). Roughly speaking, an oracle result states that prediction based on the Lasso solution
will be as accurate as if the true model was known. Variable selection properties
of the Lasso are by now also quite well understood and it was shown that consistency in
this mode can be characterized by a so called irrerepresentable condition, see for instance
Zhao and Yu (2006) or Meinshausen and Bühlmann (2006).
Nevertheless, the Lasso theory still needs to bridge a major gap with traditional statistical
inference. Indeed, it is to this date difficult to assign a measure of incertainty to its estimates.
One consequence is that the data analyst who goes beyond the goal of prediction
and is interrested in selecting the exact generating model will typically face many noise
variables for which no statistical tests are available. Also, there are no confidence intervals
for the estimated coeffficents.
Assigning confidence intervals to an estimator or testing for a null hypothesis typically
involves understanding its distributional properties. In the situation where an analytic
1