PDF of Lecture Notes - School of Mathematical Sciences

More documents

Recommendations

Info

∂l ∂θ = n − n¯xe θ ∴ ∂l ∂θ = 0 =⇒ 1 = ¯xeθ =⇒ e θ = 1¯x =⇒ ˆθ = − log ¯x. But ˆλ = 1¯x ( 1¯x) =⇒ log ˆλ = log = − log ¯x = ˆθ, as required. Remark (Not examinable) 2. STATISTICAL INFERENCE Maximum likelihood estimation & method of moments, can both be generated by the use of estimating function. An estimating function is a function, H(x; θ), with the property E{H(x; θ)} = 0. H can be used to define an estimator ˜θ which is a solution to the equation H(x; θ) = 0. For method of moments estimates, we can take H(x; θ) = ¯x − E(X) [E( ¯X) if not i.i.d. case]. To calculate the MLE: (1) find the log-likelihood, then (2) calculate the score & let it equal 0. For maximum likelihood, we can use H(x; θ) = U(θ; x) = ∂l ∂θ . Recall we showed previously that E{U(θ; x)} = 0. Asymptotic Properties of MLE’s: Suppose X 1 , X 2 , X 3 , . . . is a sequence of i.i.d. RV’s with common PDF (or probability function) f(x; θ). Assume: (1) θ 0 is the true value of the parameter θ; 101
2. STATISTICAL INFERENCE (2) f is s.t. f(x; θ 1 ) = f(x; θ 2 ) for all x =⇒ θ 1 = θ 2 . We will show (in outline) that if ˆθ n is the MLE (maximum likelihood estimator) based on X 1 , X 2 , . . . , X n , then: (1) ˆθ n → θ 0 in probability, i.e., for each ɛ > 0, lim P (|θ n − θ 0 | > ɛ) = 0. n→∞ (2) √ ni(θ0 )(ˆθ n − θ 0 ) −→ D N(0, 1) as n → ∞ Remark (Asymptotic Normality). The practical use of asymptotic normality is that for large n, ( ) 1 ˆθ ∼: N θ 0 , . ni(θ 0 ) Outline of consistency & asymptotic normality for MLE’s: Consider i.i.d. data X 1 , X 2 , . . . with common PDF/prob. function f(x; θ). If ˆθ n is the MLE based on X 1 , X 2 , . . . , X n , we will show (in outline) that ˆθ n → θ 0 in probability as n → ∞, where θ 0 is the true value for θ. Lemma. 2.3.1 Suppose f is such that f(x; θ 1 ) = f(x; θ 2 ) for all x =⇒ θ 1 = θ 2 . Then l ∗ (θ) = E{log f(x; θ)} is maximized uniquely by θ = θ 0 . Proof. l ∗ (θ) − l ∗ (θ 0 ) = = ≤ = = ∫ ∞ −∞ ∫ ∞ −∞ ∫ ∞ −∞ ∫ ∞ −∞ ∫ ∞ −∞ (log f(x; θ))f(x; θ 0 )dx − ( f(x; θ) log f(x; θ 0 ) ) f(x; θ 0 )dx ( f(x; θ) f(x; θ 0 ) − 1 ) f(x; θ 0 )dx (f(x; θ) − f(x; θ 0 ))dx f(x; θ)dx − ∫ ∞ −∞ ∫ ∞ −∞ f(x; θ 0 )dx. (log f(x; θ 0 ))f(x; θ 0 )dx 102
Page 1 and 2:
MATHEMATICAL STATISTICS III Lecture
Page 3 and 4:
1.9 Moments . . . . . . . . . . . .
Page 5 and 6:
1. DISTRIBUTION THEORY ⎧∑ h(x)p
Page 7 and 8:
n(1 − p) E(X) = , p n(1 − p) Va
Page 9 and 10:
1. DISTRIBUTION THEORY ) ( N ) p(x)
Page 11 and 12:
1. DISTRIBUTION THEORY that is, ⎧
Page 13 and 14:
1. DISTRIBUTION THEORY 3. Gamma (K,
Page 15 and 16:
1. DISTRIBUTION THEORY Figure 9: Ca
Page 17 and 18:
1. DISTRIBUTION THEORY F Y (y) = P
Page 19 and 20:
1. DISTRIBUTION THEORY Solution. Ob
Page 21 and 22:
1. DISTRIBUTION THEORY 1.6 Moments
Page 23 and 24:
1. DISTRIBUTION THEORY where φ(a)
Page 25 and 26:
1. DISTRIBUTION THEORY Examples 1.
Page 27 and 28:
1. DISTRIBUTION THEORY Possible val
Page 29 and 30:
1. DISTRIBUTION THEORY Remarks 1. O
Page 31 and 32:
1. DISTRIBUTION THEORY Figure 12: A
Page 33 and 34:
1. DISTRIBUTION THEORY Solution. Re
Page 35 and 36:
⎧ ⎪⎨ 1 0 < x 2 < 1 f 2 (x 2 )
Page 37 and 38:
1. DISTRIBUTION THEORY Just substit
Page 39 and 40:
1. DISTRIBUTION THEORY 2. Suppose Z
Page 41 and 42:
1. DISTRIBUTION THEORY Solution. St
Page 43 and 44:
1. DISTRIBUTION THEORY Recall stand
Page 45 and 46:
∂ ∂r g 1(r, θ) = ∂ r cos θ
Page 47 and 48:
1. DISTRIBUTION THEORY ⇒ det(G) =
Page 49 and 50:
1. DISTRIBUTION THEORY Definition.
Page 51 and 52:
1. DISTRIBUTION THEORY Proof. Consi
Page 53 and 54: 1. DISTRIBUTION THEORY Proof. 1. (C
Page 55 and 56: 1. DISTRIBUTION THEORY Proof. M X (
Page 57 and 58: 1. DISTRIBUTION THEORY Proof. M AX+
Page 59 and 60: 1. DISTRIBUTION THEORY Remark This
Page 61 and 62: 1. DISTRIBUTION THEORY 2. If Z 1 ,
Page 63 and 64: 1. DISTRIBUTION THEORY where X i ,
Page 65 and 66: 1. DISTRIBUTION THEORY ⎧ ⎪⎨
Page 67 and 68: 1. DISTRIBUTION THEORY If we take Z
Page 69 and 70: =⇒ Var(X) = Σ = =⇒ Σ −1 = =
Page 71 and 72: 1. DISTRIBUTION THEORY 2. Independe
Page 73 and 74: 1. DISTRIBUTION THEORY and (R 1 ) n
Page 75 and 76: 1. DISTRIBUTION THEORY Figure 16: D
Page 77 and 78: 1. DISTRIBUTION THEORY Remarks This
Page 79 and 80: 1. DISTRIBUTION THEORY Hence we hav
Page 81 and 82: 2. STATISTICAL INFERENCE Definition
Page 83 and 84: 2. STATISTICAL INFERENCE Figure 19:
Page 85 and 86: 2. STATISTICAL INFERENCE Remark If
Page 87 and 88: 2. STATISTICAL INFERENCE Finally ob
Page 89 and 90: 2. STATISTICAL INFERENCE Proof. (1)
Page 91 and 92: 2. STATISTICAL INFERENCE Hence the
Page 93 and 94: 2. STATISTICAL INFERENCE =⇒ f is
Page 95 and 96: 2. STATISTICAL INFERENCE Examples (
Page 97 and 98: 2. STATISTICAL INFERENCE According
Page 99 and 100: 2. STATISTICAL INFERENCE p = 2 =⇒
Page 101 and 102: 2. STATISTICAL INFERENCE To find ˆ
Page 103: 2. STATISTICAL INFERENCE If X ∼Ex
Page 107 and 108: 2. STATISTICAL INFERENCE Then, U n
Page 109 and 110: 2. STATISTICAL INFERENCE and the al
Page 111 and 112: 2. STATISTICAL INFERENCE =⇒ W =
Page 113 and 114: 2. STATISTICAL INFERENCE Recall Wal
Page 115 and 116: 2. STATISTICAL INFERENCE Let C 1 =
Page 117 and 118: 2. STATISTICAL INFERENCE Figure 23:
show all

PDF of Lecture Notes - School of Mathematical Sciences

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?