Student Notes To Accompany MS4214: STATISTICAL INFERENCE

More documents

Recommendations

Info

Differentiating again, and multiplying by −1 yields the information function Clearly ˆυ is the MLE since Define I(υ) = − n 1 + 2υ2 υ3 n� (xi − µ) 2 . i=1 I(ˆυ) = n > 0. 2υ2 Zi = (Xi − µ) 2 / √ υ, so that Zi ∼ N(0, 1). From the appendix on probability n� Z 2 i ∼ χ 2 n, implying E[ � Z 2 i ] = n, and Var[ � Z 2 i ] = 2n. The MLE Then and Finally, Var[ˆυ] = i=1 ˆυ = (υ/n) E[ˆυ] = E � υ n n� i=1 n� i=1 Z 2 i Z 2 i . � = υ, � � υ � n� 2 Var Z n i=1 2 � i = 2υ2 n . E [I(υ)] = − n 1 + 2υ2 υ3 = − n nυ + 2υ2 υ3 = n . 2υ2 n� E � (xi − µ) 2� Hence the CRLB = 2υ 2 /n, and so ˆυ has efficiency 1. � Our treatment of the two parameters of the Gaussian distribution in the last example was to (i) fix the variance and estimate the mean using maximum likelihood; and then (ii) fix the mean and estimate the variance using maximum likelihood. In practice we would like to consider the simultaneous estimation of these parameters. In the next section of these notes we extend MLE to multiple parameter estimation. 27 i=1
2.5 Multi-parameter Estimation Suppose that a statistical model specifies that the data y has a probability distribution f(y; α, β) depending on two unknown parameters α and β. In this case the likelihood function is a function of the two variables α and β and having observed the value y is defined as L(α, β) = f(y; α, β) with ℓ(α, β) = ln L(α, β). The MLE of (α, β) is a value (ˆα, ˆ β) for which L(α, β) , or equivalently ℓ(α, β) , attains its maximum value. Define S1(α, β) = ∂ℓ/∂α and S2(α, β) = ∂ℓ/∂β. The MLEs (ˆα, ˆ β) can be obtained by solving the pair of simultaneous equations : S1(α, β) = 0 S2(α, β) = 0 The information matrix I(α, β) is defined to be the matrix : � I11(α, β) I(α, β) = I21(α, β) � � I12(α, β) = − I22(α, β) ∂2 ∂α2 ℓ ∂2 ∂α∂β ℓ ∂2 ∂β∂α ℓ ∂2 ∂β2 ℓ The conditions for a value (α0, β0) satisfying S1(α0, β0) = 0 and S2(α0, β0) = 0 to be a MLE are that and I11(α0, β0) > 0, I22(α0, β0) > 0, det(I(α0, β0) = I11(α0, β0)I22(α0, β0) − I12(α0, β0) 2 > 0. This is equivalent to requiring that both eigenvalues of the matrix I(α0, β0) be positive. Example 2.10 (Gaussian distribution). Let X1, X2 . . . , Xn be iid observations from a N (µ, v) density in which both µ and v are unknown. The log likelihood is ℓ(µ, v) = n� � 1 ln √ exp [− 2πv i=1 1 2v (xi − µ) 2 = � ] n� � − i=1 1 1 1 ln [2π] − ln [v] − 2 2 2v (xi − µ) 2 � = − n n� n 1 ln [2π] − ln [v] − (xi − µ) 2 2 2v 2 . Hence implies that S1(µ, v) = ∂ℓ ∂µ ˆµ = 1 n = 1 v i=1 n� (xi − µ) = 0 i=1 n� xi = ¯x. (2.4) i=1 28 �
Page 1 and 2: Student Notes To Accompany MS4214:
Page 3 and 4: 4.8 Worked Problems . . . . . . . .
Page 5 and 6: Suppose that, in order to learn som
Page 7 and 8: Example 1.4 (Blood pressure). We wi
Page 9 and 10: Chapter 2 The Theory of Estimation
Page 11 and 12: Thus if we carried out an infinite
Page 13 and 14: Problem 2.1. Let X have a binomial
Page 15 and 16: Problem 2.6. Let X1, . . . , Xn be
Page 17 and 18: Theorem 2.8 (Cramér Rao lower boun
Page 19 and 20: Let us propose ˆµ = ¯ X as an es
Page 21 and 22: Example 2.2 (Bernoulli Trials). Con
Page 23 and 24: Relative Likelihood 0.0 0.2 0.4 0.6
Page 25 and 26: Example 2.7 (Exponential distributi
Page 27: ℓ(µ) = −m ln µ − 1 m� ti
Page 31 and 32: Lemma 2.9 (Joint distribution of th
Page 33 and 34: So, if | ˆ θ − θ0| is small, w
Page 35 and 36: Then the log-likelihood function is
Page 37 and 38: A regression of the empirical distr
Page 39 and 40: 2.7 The Invariance Principle How do
Page 41 and 42: Event Probability Set 0 0 0 (1 −
Page 43 and 44: 2.10 Worked Problems The Problems 1
Page 45 and 46: (a) Show that ˆµ is an unbiased e
Page 47 and 48: 4. E(X2 ) = � ∞ 0 x2f(x)dx =
Page 49 and 50: Student Questions 1. Let X1, X2, .
Page 51 and 52: Chapter 3 The Theory of Confidence
Page 53 and 54: Suppose that we have data X1, X2, .
Page 55 and 56: Lemma 3.2 (The Student t-distributi
Page 57 and 58: Example 3.6. Suppose that we have d
Page 59 and 60: 3.3 Approximate Confidence Interval
Page 61 and 62: 3.4 Worked Problems The Problems 1.
Page 63 and 64: Student Questions 1. Let X1, X2, .
Page 65 and 66: Chapter 4 The Theory of Hypothesis
Page 67 and 68: Example 4.3 (The power function). S
Page 69 and 70: (d) Suppose Θ0 = {(µ, σ) : −
Page 71 and 72: (c) Θ0 = {(µ, µ,σ) : −∞ <
Page 73 and 74: The Score Test Statistic: This test
Page 75 and 76: 4.5 The Neyman-Pearson Lemma Suppos
Page 77 and 78: According to the Neyman-Pearson lem
Page 79 and 80:
would consider to be an unusually l
Page 81 and 82:
pendent we would expect the proport
Page 83 and 84:
3. Explain what is meant by the pow
Page 85 and 86:
For the null hypothesis θ = 1, the
Page 87 and 88:
0.1820. Similarly P (X = 3) = 0.218
Page 89 and 90:
Appendix A Review of Probability A.
Page 91 and 92:
where Xi ∼ Bernoulli(θ) for i =
Page 93 and 94:
The expected value of the jth momen
Page 95 and 96:
A.3 Continuous Random Variables A.3
Page 97 and 98:
A.3.4 Gaussian Distribution A rando
Page 99 and 100:
A.3.5 Weibull Distribution The Weib
Page 101 and 102:
Next, let g(y) be a monotone decrea
Page 103 and 104:
That is, X + Y ∼ Pois(θ + λ). =
Page 105 and 106:
since � ∞ −∞ e−αu2 du =
Page 107 and 108:
A.4.4 The Bivariate Normal Distribu
Page 109 and 110:
A.5 Generating Functions Denote the
Page 111 and 112:
Density p.g.f. m.g.f. ch.f. c.g.f B
Page 113:
Uniform U(a, b) Continuous Distribu
show all

Student Notes To Accompany MS4214: STATISTICAL INFERENCE

Create successful ePaper yourself

Delete template?

Save as template?