Subsampling estimates of the Lasso distribution.

More documents

Recommendations

Info

2.1 Convergence in probability 5 for every x ∈ C, where f is a real (random) variable on C. Further, assume that f is uniquely minimized at α ∈ C and let {α n } n be a sequence of minimizers of {f n } n , i.e. α n ∈ arg min x∈C f n (·), n ∈ N. Then, α n → P α. Proof. The proof rests on (Hjort and Pollard, 1993, Lemma 2) and subsequent remarks. For arbitrary δ > 0, define and ∆ n (δ) = h(δ) = sup |f n (x) − f(x)| ‖x−α‖≤δ inf f n(x) − f n (α) ‖x−α‖=δ First, we show that P (‖α n − α‖ ≥ δ) ≤ P (∆ n (δ) ≥ 1 2 h(δ) ) . (2.1.0.2) For every x ∈ R p with ‖x − α‖ > δ there exist some c with ‖c − α‖ = δ and t ∈ [0, 1] such that c = tx + (1 − t)α. By convexity of the functions f n , we have f n (c) ≤ tf n (x) + (1 − t)f n (α). for every n ∈ N. Writing r n (x) = f(x) − f n (x), we obtain t(f n (x) − f n (α)) ≥ f n (c) − f n (α) = f(c) − f(α) + r n (c) − r n (α) ≥ h(δ) − 2∆ n (δ) In particular, {∆ n (δ) < 1 } 2 h(δ) ⊆ {f n (x) > f n (α)} for every x with ‖x − α‖ > δ and every n ∈ N. Since α minimizes f n , this implies that {‖α n − α‖} ⊆ {∆ n (δ) ≥ 1 } 2 h(δ) and 2.1.0.2 follows. Next, we show that ∆ n (δ) = o P (1) (2.1.0.3)
6 Minimizers of convex processes Let ε > 0, for arbitrary M > 0 we have P (∆ n (δ) > ε) = P (∆ n (δ) > ε; ‖α‖ ≤ M) + P (∆ n (δ) > ε; ‖α‖ > M) ( ) ≤ P sup |f n (x) − f(x)| > ε ‖x‖≤δ+M = o(1) + P (‖α‖ > M) + P (‖α‖ > M) by Theorem 2.1.0.2. Letting M tend to infitnity completes the argument. Now, we can prove α n → P α. Let δ > 0, for fxed arbitrary M > 0, we have P (‖α n −α‖ > δ) ≤ P (∆ n (δ) ≥ 1 ) 2 h(δ) ( = P ∆ n (δ) ≥ 1 ) ( 2 h(δ); 1 h(δ) ≤ M + P ∆ n (δ) ≥ 1 ) 2 h(δ); 1 h(δ) > M ( ≤ P ∆ n (δ) ≥ 1 ) ( ) 1 + P 2M h(δ) > M For fixed M, the first term tends to zero as n tends to infinity by 2.1.0.3. Then, the second term tends to zero as M tends to infinity. Indeed, h(δ) −1 is almost surely finite by uniqueness of α. This completes the proof. 2.2 Convergence in distribution The goal of this section is to derive conditions under which a sequence of minimizers of convex objective functions converge in distribution and to provide means to determine this limit. At first, the concept of weak convergence must be revisited to encompass not necessarily measurable maps defined on probability spaces, this is a feature typically exhibited by argmin functionals. This wider concept of weak convergence was originally introduced by Hofmann-Jørgensen but first exposited in Dudley (1985) and Pollard (1990). However, in the present section we follow the more mature exposition of Van der Vaart and Wellner (1996). Indeed they showed that even in this general setting, most important results from weak convergence theory, going from the portmanteau theorem to the almost sure representation theorem, through Prohorov’s theorem, remain valid, provided one makes necessary, but essentially minor, modifications. This section is ended with the argmin continuous mapping theorem which will be applied to the Lasso estimator in the next chapter. Definition 2.2.0.4. Let (Ω, A, P ) be a probability space and T : Ω → R be an arbitrary map. (i) The outer expection and the inner expectation of T with respect to P are defined as E ∗ (T ) = inf {E(U)|U ≥ T, U : Ω → R measurable and E(U) exists} (2.2.0.4)
Page 1: ✄✄ ✄ ✄ ✄ ✄✄ ✄ ✄
Page 4 and 5: iv Abstract
Page 6 and 7: vi CONTENTS Contents Notation viii
Page 8 and 9: viii Notation List of Tables 6.1 Mo
Page 10 and 11: Chapter 1 Introduction In this thes
Page 12 and 13: Chapter 2 Minimizers of convex proc
Page 16 and 17: 2.2 Convergence in distribution 7 a
Page 18 and 19: 2.2 Convergence in distribution 9 L
Page 20 and 21: 2.2 Convergence in distribution 11
Page 22 and 23: 2.2 Convergence in distribution 13
Page 24 and 25: 2.3 A continous mapping theorem for
Page 30 and 31: Chapter 3 Application to the Lasso
Page 32 and 33: 3.2 Limit in distribution 23 almost
Page 34 and 35: 3.2 Limit in distribution 25 We als
Page 36 and 37: 3.2 Limit in distribution 27 3.2.2
Page 38 and 39: 3.2 Limit in distribution 29 Figure
Page 40 and 41: Chapter 4 The adaptive Lasso in a h
Page 42 and 43: 33 B.3 (Adaptive irrepresentable co
Page 44 and 45: 35 exists a constant K d depending
Page 46 and 47: 4.1 Variable selection consistency
Page 48 and 49: 4.2 Partial asymptotic normality 39
Page 50 and 51: 4.3 Marginal regressors as initial
Page 52 and 53: Chapter 5 Subsampling In his semina
Page 54 and 55: 5.1 Pointwise consistency for distr
Page 56 and 57: 5.2 Uniform consistency for quantil
Page 64 and 65:
Chapter 6 Numerical results Motivat
Page 66 and 67:
6.1 Low dimensinal setting 57 5. De
Page 68 and 69:
6.1 Low dimensinal setting 59 Figur
Page 70 and 71:
6.2 High dimensional setting 61 3.
Page 72 and 73:
6.2 High dimensional setting 63 (d)
Page 74 and 75:
6.2 High dimensional setting 65 Mod
Page 76 and 77:
6.2 High dimensional setting 67 Mod
Page 78 and 79:
6.2 High dimensional setting 69 Fig
Page 80 and 81:
Chapter 7 Concluding remarks We rev
Page 82 and 83:
Bibliography Andersen, P. and R. Gi
Page 84 and 85:
Appendix A R Codes A.1 Simulation c
Page 86 and 87:
A.1 Simulation code for the Lasso i
Page 88 and 89:
A.2 Simulation code for the adaptiv
Page 90 and 91:
A.2 Simulation code for the adaptiv
show all

Subsampling estimates of the Lasso distribution.

Create successful ePaper yourself

Delete template?

Save as template?