Subsampling estimates of the Lasso distribution.

Chapter 4
The adaptive Lasso in a high
dimensional setting
In a high dimensional setting, where the use of the Lasso is most justified, due to its
sparsity inducing property, there are to this date no asymptotic results similar to those of
Knight and Fu (2000). Hence we turn to a variant, the adaptive Lasso and presents the
results of Huang et al. (2008) who studied it under saprsity asssumptions and a further a
partial orthogonality assumption between relevant and noise covariates. Their approach
offers the advantage to provide an asymptotic normality result for estimates corresponding
to nonzero-coefficients.
One typically resorts to a triangular array to model high dimensionality in linear models,
that is, one assumes that
Y i = x ′ iβ n0 + ε i , i = 1, . . . , n (4.0.2.1)
where the parameter β n0 has a dimension p n allowed to grow faster than n. For observations
(Y i , x i ), i = 1, . . . , n drawn from 4.0.2.1, the adaptive Lasso is defined as
arg min
φ∈R p
L n ( φ) = arg min
φ∈R p
n ∑
i=1
(Y i − x ′ iφ) 2 ∑p n
+ λ n w nj |φ j | (4.0.2.2)
where λ n is the usual penalty parameter and are {w nj } j are in general strictly positive
weights. However, here we make the assumption that they are computed using an initial
estimator ˜β nj of β nj , that is :
j=1
w nj = | ˜β nj | −1 , (4.0.2.3)
for j = 1, · · · , n. For notational convenience, in the remaining, we drop the subscript n
for β n0 , yet dependence on n is implicitely assumed.
Next, we introduce further notation.
First, assume without loss of generality that β 0 takes the form
β 0 = (β ′ 1, β ′ 2) ′
31