Subsampling estimates of the Lasso distribution.

More documents

Recommendations

Info

Chapter 2 Minimizers of convex processes This chapter is devoted to the asymptotic theory of minimizers of convex processes. The general message we hope to convey is that both in probability and in distribution, the determination of the limit for such minimizers can be reduced to the determination of the pointwise (or marginal) limit of the processes, provided that some regularity conditions are met and that this pointwise limiting process is uniquely minimized. 2.1 Convergence in probability We first show that, in probability, pointwise convergence of convex processes on a dense subset implies uniform convergence on compacts. This result will be derived from the following determistic version after a subsequencing argument. Theorem 2.1.0.1. (Rockafellar, 1970, Theorem 10.8) Let C ⊆ R p be open and let {f n : C → R} n be a sequence of finite convex functions on C. Suppose that {f n } n converges pointwise on a dense subset of C 0 of C, that is lim f n(x) n→∞ exists for every x ∈ C 0 . Then, {f n } n converges pointwise on the whole set C and the limit function f(x) := lim n→∞ f n(x) is finite and convex on C. Moreover, {f n } n∈N converges uniformly to f on every compact subset K ⊂ C, that is sup |f n (x) − f(x)| = 0. x∈K Theorem 2.1.0.2. (Andersen and Gill, 1982, Theorem 2.1) Let C ⊆ R p be open and let {f n } n be a sequence of random convex functions on C such that for every x ∈ C, f n (x) → P f(x), where f is a real (random) function on C. Then f is convex and for every compact subset K ⊂ C, it holds that sup |f n (x) − f(x)| → P 0. x∈K 3
4 Minimizers of convex processes Proof. Recall that convergence in probability implies almost sure convergence along some subsequence. Consider a dense sequence {x i } i ⊂ C. Since f n (x i ) → P f(x i ) for every i ∈ N, there exists a subsequence {j 1 n} n for which f j 1 n (x 1 ) → f(x 1 ) (2.1.0.1) almost everywhere. almost everywhere, This subsequence contains a further subsequence {j 2 n} n for which, f j 2 n (x 2 ) → f(x 2 ) holds in addition to 2.1.0.1. subsequence {jn} k n satisfying Repeating this argument, we obtain for every k ∈ N a f j k n (x i ) → f(x i ) for i = 1, . . . , k, almost everywhere. Then for the sequence {j ∗ n} n of diagonal members, i.e. j ∗ n = j n n, we have f j ∗ n (x i ) → f(x i ) for every i ∈ N almost surely. Next, Theorem 2.1.0.1 implies that f is almost everywhere convex on C and that ∣ sup ∣f j ∗ n (x) − f(x) ∣ → 0 x∈K almost surely for every compact K ⊂ C. We have shown more generally that every subsequence of {sup x∈K |f n (x) − f(x)|} n has a further subsequence which converges almost surely to zero. This implies convergence in probability to zero along the whole sequence. Indeed, suppose that this is not the case, then there exists a subsequence {j n } n and costants M, ε > 0 such that P (sup x∈K |f jn (x) − f(x)| > M) > ε for all n ∈ N. For every further subsequence {j n} ′ n , it holds that ( ∣ P ∣f j ′ n (x) − f(x) ∣ ) ⎛ { ∞⋃ ⋂ ∣ → 0 ≤ P ⎝ ∣f j ′ n (x) − f(x) ∣ ≤ M} ⎞ ⎠ sup x∈K N=1 n≥N sup x∈K ( ) ≤ lim P ∣ sup ∣f n→∞ j ′ N (x) − f(x) ∣ ≤ M ≤ 1 − ε which stands in contradiction with the previous conclusion. This argument is known as the subsequence criterion((Kallenberg, 2002, Lemma 3.2)). Remark. Note that a more direct and elementary proof of the previous result was given by (Pollard, 1991, Convexity Lemma). x∈K Next, we use this result to infer on the minimizers themselves. Corollary 2.1.0.3. Let C ⊆ R be open and let {f n } n be a sequence of random convex functions on C such that f n (x) → P f(x)
Page 1: ✄✄ ✄ ✄ ✄ ✄✄ ✄ ✄
Page 4 and 5: iv Abstract
Page 6 and 7: vi CONTENTS Contents Notation viii
Page 8 and 9: viii Notation List of Tables 6.1 Mo
Page 10 and 11: Chapter 1 Introduction In this thes
Page 14 and 15: 2.1 Convergence in probability 5 fo
Page 16 and 17: 2.2 Convergence in distribution 7 a
Page 18 and 19: 2.2 Convergence in distribution 9 L
Page 20 and 21: 2.2 Convergence in distribution 11
Page 22 and 23: 2.2 Convergence in distribution 13
Page 24 and 25: 2.3 A continous mapping theorem for
Page 30 and 31: Chapter 3 Application to the Lasso
Page 32 and 33: 3.2 Limit in distribution 23 almost
Page 34 and 35: 3.2 Limit in distribution 25 We als
Page 36 and 37: 3.2 Limit in distribution 27 3.2.2
Page 38 and 39: 3.2 Limit in distribution 29 Figure
Page 40 and 41: Chapter 4 The adaptive Lasso in a h
Page 42 and 43: 33 B.3 (Adaptive irrepresentable co
Page 44 and 45: 35 exists a constant K d depending
Page 46 and 47: 4.1 Variable selection consistency
Page 48 and 49: 4.2 Partial asymptotic normality 39
Page 50 and 51: 4.3 Marginal regressors as initial
Page 52 and 53: Chapter 5 Subsampling In his semina
Page 54 and 55: 5.1 Pointwise consistency for distr
Page 56 and 57: 5.2 Uniform consistency for quantil
Page 62 and 63:
5.2 Uniform consistency for quantil
Page 64 and 65:
Chapter 6 Numerical results Motivat
Page 66 and 67:
6.1 Low dimensinal setting 57 5. De
Page 68 and 69:
6.1 Low dimensinal setting 59 Figur
Page 70 and 71:
6.2 High dimensional setting 61 3.
Page 72 and 73:
6.2 High dimensional setting 63 (d)
Page 74 and 75:
6.2 High dimensional setting 65 Mod
Page 76 and 77:
6.2 High dimensional setting 67 Mod
Page 78 and 79:
6.2 High dimensional setting 69 Fig
Page 80 and 81:
Chapter 7 Concluding remarks We rev
Page 82 and 83:
Bibliography Andersen, P. and R. Gi
Page 84 and 85:
Appendix A R Codes A.1 Simulation c
Page 86 and 87:
A.1 Simulation code for the Lasso i
Page 88 and 89:
A.2 Simulation code for the adaptiv
Page 90 and 91:
A.2 Simulation code for the adaptiv
show all

Subsampling estimates of the Lasso distribution.

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?