STATISTICS 512 TECHNIQUES OF MATHEMATICS FOR ...

More documents

Recommendations

Info

32 • Return to regression. (i) Least squares estimation in terms of hat matrix decomposition of norm of residuals: Note that x ⊥ y⇒kx + yk 2 = kxk 2 + kyk 2 ;then ° °y − Xˆθ ° 2 ° = ° °°H ³ Xˆθ´° y ° 2 ° − + °°(I ³ Xˆθ´° − H) y ° 2 − = ° °H ³ y Xˆθ´° ° 2 − + k(I − H) yk 2 ≥ k(I − H) yk 2 with equality iff Hy = Xˆθ iff (‘if and only if’) ˆθ = ³ X 0 X´−1 X 0 y (how?), the LS estimator. The fitted values are ŷ = Xˆθ = Hy and are orthogonal to the residuals e = y − ŷ =(I − H) y We say that H and I − H project the data (y) onto the estimation space and error space, respectively, and that these spaces are orthogonal.
33 (ii) In terms of QR-decomposition: we have that ˆθ = R −1 1 i.e. ˆθ is the solution to Thus compute z ×1 = Q 0 y = = Ã ! Q 0 1 y Q 0 2 y ³ R 0 1´−1 R 0 1 Q 0 1 y; R 1ˆθ = Q 0 1 y Ã Q 0 1 Q 0 2 ! y = Ã z1 z 2 ! × 1 ( − ) × 1 Then backsolve the system of equations R 1ˆθ = z 1 . Numerically stable - no matrix inversions. • The residual vector is e = Q 2 z 2 , with squared norm kz 2 k 2 . The usual estimate of the variance 2 of the random errors is ˆ 2 = SS of residuals − = kek2 − = kz 2k 2 −
Page 1 and 2: STATISTICS 512 TECHNIQUES OF MATHEM
Page 3 and 4: II LIMITS, CONTINUITY, DIFFEREN- TI
Page 5 and 6: 19 Numerical optimization: Steepest
Page 7 and 8: 7 1. Introduction; matrix manipulat
Page 9 and 10: - is of rather limited usefulness.
Page 11 and 12: 11 — Block matrices ... a particu
Page 13 and 14: 13 for a random matrix X. You shoul
Page 15 and 16: 15 Here the observations (rows) hav
Page 17 and 18: 17 3. Identity element: There is 0
Page 19 and 20: 19 — Fact 1: Every vector space h
Page 21 and 22: 21 3) Used often: (A 0 A)=(A). Proo
Page 23 and 24: 23 areallsolvable. Wewrite[b 1 ··
Page 25 and 26: 25 • Angle between nonzero vecto
Page 27 and 28: 27 (why?). Geometrically, an orthog
Page 29 and 30: 29 orthogonal unit vectors q 1 q h
Page 31: 31 4. LSEs; Spectral theory • Rec
Page 35 and 36: 35 and = Ã P ⎛ ⎜ ⎝ R S Q !
Page 37 and 38: 37 • Now suppose that M is symmet
Page 39 and 40: 39 • Spectral Decomposition Theor
Page 41 and 42: 41 5. Examples & applications Conse
Page 43 and 44: 43 This is the ellipsoid in R with
Page 45 and 46: 45 • If H is idempotent then (i)
Page 47 and 48: 47 and similarly define 2 2 as the
Page 49 and 50: 49 Application 2. By the Cauchy-Sch
Page 51 and 52: 51 6. Limits; continuity; probabili
Page 53 and 54: 53 — ⊂ R is closed if it cont
Page 55 and 56: 55 bounded, q say =(0). It can be
Page 57 and 58: 57 7. Random variables; distributio
Page 59 and 60: 59 • Since the set = (−∞] is
Page 61 and 62: 61 • Jensen’s Inequality: If :
Page 63 and 64: 63 • If → and the function
Page 65 and 66: 65 • Linearity, product, quotient
Page 67 and 68: 67 • Taylor’s Theorem:“Suffic
Page 69 and 70: 69 We want to show that (), which
Page 71 and 72: 71 9. Applications: transformations
Page 73 and 74: 73 monotonic. Example: suppose ∼
Page 75 and 76: 75 Now suppose that → and (ac
Page 77 and 78: 77 This can make it problematic to
Page 79 and 80: 79 10. Sequences and series • Con
Page 81 and 82: 81 Then for each , sup | () − ()
Page 83 and 84:
83 Thus a convergent sequence (i.e.
Page 85 and 86:
85 Then for 0 we have 0 = + X
Page 87 and 88:
87 11. Power series; moment and pro
Page 89 and 90:
89 • If P ∞ =0 converges fo
Page 91 and 92:
91 • By (ii), we can repeat the p
Page 93 and 94:
93 • The moment generating functi
Page 95 and 96:
95 12. Branching processes • Impo
Page 97 and 98:
97 Considering the probabilities of
Page 99 and 100:
99 Since = ( −1 )and is continu
Page 101 and 102:
101 P(N
Page 103 and 104:
103 Then clearly () ≤ () ≤
Page 105 and 106:
105 • Monotonic, bounded function
Page 107 and 108:
107 • Now define () = Z ()
Page 109 and 110:
109 • Improper Riemann integrals,
Page 111 and 112:
111 14. Riemann and Riemann-Stieltj
Page 113 and 114:
113 • A generalization of the Rie
Page 115 and 116:
115 • Improper R-S integrals defi
Page 117 and 118:
117 • Cauchy-Schwarz inequality:
Page 119 and 120:
119 constant’) ∈ [0 1) as →
Page 121 and 122:
15. Moment generating functions; Ch
Page 123 and 124:
123 • Suppose ∼ (0 1), with p.
Page 125 and 126:
125 • Chebyshev’s Inequality fu
Page 127 and 128:
127 Let be fixed but arbitrary. Ex
Page 129 and 130:
129 • Slutsky’s Theorem: If
Page 131 and 132:
131 Part IV MULTIDIMENSIONAL CALCUL
Page 133 and 134:
133 • Derivatives. Put e = (00 1
Page 135 and 136:
135 — If = 1 then the Jacobian m
Page 137 and 138:
137 1. If the partial derivatives o
Page 139 and 140:
139 quantile function. Differentiat
Page 141 and 142:
141 17. Implicit Function Theorem;
Page 143 and 144:
143 • The Inverse Function Theore
Page 145 and 146:
145 • Example. Write the characte
Page 147 and 148:
147 • Often we seek extrema of mu
Page 149 and 150:
149 under which the satisfaction of
Page 151 and 152:
151 Since x 2 is a stationary point
Page 153 and 154:
153 If sup or equivalently if ()
Page 155 and 156:
155 Here and elsewhere the assumpti
Page 157 and 158:
157 • Example: Let be independe
Page 159 and 160:
159 order that 2 () integrate to 1
Page 161 and 162:
161 Note that x ¯ ¯ = ¯¯¯¯¯
Page 163 and 164:
163 with = 0 to minimize (by tria
Page 165 and 166:
165 —Example:solve () =log − 1=
Page 167 and 168:
167 regression, so I’ll illustrat
Page 169 and 170:
169 Assuming convergence, the limit
Page 171 and 172:
171 • For i.i.d. observations wit
Page 173 and 174:
173 • The MLE is generally obtain
Page 175 and 176:
175 so that ³ covθ h˙(θ) i =´
Page 177 and 178:
177 We have (by the WLLN) that X Ã
Page 179 and 180:
with gradient ˙(θ) = Ã − +
Page 181 and 182:
181 • Method of moments: Define p
Page 183 and 184:
• The limit of the NR-process is
Page 185 and 186:
185 • Now let (X) be any unbiased
Page 187 and 188:
187 22. Minimax M-estimation I •
Page 189 and 190:
189 • The asymptotic variance doe
Page 191 and 192:
191 • We will show that such a pa
Page 193 and 194:
193 • By the Lemma, () in(22.3)is
Page 195 and 196:
195 • By comparison with (22.4) w
Page 197 and 198:
We note that and () = − 0 = −
Page 199 and 200:
199 • A solution (there are three
Page 201 and 202:
• The asymptotic normality result
Page 203 and 204:
203 where the infimum is over all s
Page 205 and 206:
205 ∪ =1 ;inthiscasedefine R =
Page 207 and 208:
207 • If is Lebesgue measure the
Page 209 and 210:
209 • The expected value of a r.v
Page 211:
211 • It was stated above that if
show all

STATISTICS 512 TECHNIQUES OF MATHEMATICS FOR ...

Create successful ePaper yourself

Delete template?

Save as template?