Student Notes To Accompany MS4214: STATISTICAL INFERENCE

More documents

Recommendations

Info

Reparameterization Negative parameter estimates can be avoided by reparameterizing the profile log- likelihood in (2.11) using α = ln(a). Since a = e α we are guaranteed to obtain a > 0. The reparameterized profile log-likelihood becomes � n� 1 ℓα(α) = mα − m ln t m eα � i + (e α − 1) implying score function and information function I(α) = i=1 S(α) = m − meα � n meα �n i=1 teα i i=1 teα �n i=1 teα i � n� t eα i ln(ti) − e α [� n i=1 i ln(ti) + e α +e α m� ln (ti) − m, i=1 m� ln(ti) i=1 i=1 teα i ln(ti)] 2 �n i=1 teα i n� i=1 t eα i [ln(ti)] 2 � − e α m� ln(ti). The estimates â = 1.924941 and ˆ b = 78.12213 were obtained by applying this method to the Weibull data using starting values a0 = 0.07 and a0 = 76 in 103 and 105 iterations respectively. However, the starting values a0 = 0.06 and a0 = 77 failed due to division by computationally tiny (1.0e-300) values. The step-halving scheme The Newton-Raphson method uses the (first and second) derivatives of ℓ(θ) to max- imize the function ℓ(θ), but the function itself is not used in the algorithm. The log-likelihood can be incorporated into the Newton-Raphson method by modifying the updating step to θi+1 = θi + λiI(θi) −1 S(θi), (2.12) where the search direction has been multiplied by some λi ∈ (0, 1] chosen so that the inequality ℓ � θi + λiI(θi) −1 S(θi) � > ℓ (θi) (2.13) holds. This requirement protects the algorithm from converging towards minima or saddle points. At each iteration the algorithm sets λi = 1, and if (2.13) does not hold λi is replaced with λi/2. The process is repeated until the inequality in (2.13) is satisfied. At this point the parameter estimates are updated using (2.12) with the value of λi for which (2.13) holds. If the function ℓ(θ) is concave and unimodal convergence is guaranteed. Finally, when maxima is guaranteed, even if ℓ(θ) is not concave. Ī(θ) is used in place of I(θ) convergence to a (local) 37 i=1
2.7 The Invariance Principle How do we deal with parameter transformation? We will assume a one-to-one transformation, but the idea applied generally. Consider a binomial sample with n = 10 independent trials resulting in data x = 8 successes. The likelihood ratio of θ1 = 0.8 versus θ2 = 0.3 is L(θ1 = 0.8) L(θ2 = 0.3) = θ8 1(1 − θ1) 2 θ8 = 208.7 , 2(1 − θ2) 2 that is, given the data θ = 0.8 is about 200 times more likely than θ = 0.3. Suppose we are interested in expressing θ on the logit scale as ψ ≡ ln{θ/(1 − θ)} , then ‘intuitively’ our relative information about ψ1 = ln(0.8/0.2) = 1.29 versus ψ2 = ln(0.3/0.7) = −0.85 should be L ∗ (ψ1) L ∗ (ψ2) = L(θ1) L(θ2) = 208.7 . That is, our information should be invariant to the choice of parameterization. ( For the purposes of this example we are not too concerned about how to calculate L ∗ (ψ). ) Theorem 2.10 (Invariance of the MLE). If g is a one-to-one function, and ˆ θ is the MLE of θ then g( ˆ θ) is the MLE of g(θ). Proof. This is trivially true as we let θ = g −1 (µ) then f{y|g −1 (µ)} is maximized in µ exactly when µ = g( ˆ θ). When g is not one-to-one the discussion becomes more subtle, but we simply choose to define ˆgMLE(θ) = g( ˆ θ) It seems intuitive that if ˆ θ is most likely for θ and our knowledge (data) remains unchanged then g( ˆ θ) is most likely for g(θ). In fact, we would find it strange if ˆ θ is an estimate of θ, but ˆ θ 2 is not an estimate of θ 2 . In the binomial example with n = 10 and x = 8 we get ˆ θ = 0.8, so the MLE of g(θ) = θ/(1 − θ) is g( ˆ θ) = ˆ θ/(1 − ˆ θ) = 0.8/0.2 = 4. This convenient property is not necessarily true of other estimators. For example, if ˆ θ is the MVUE of θ, then g( ˆ θ) is generally not MVUE for g(θ). Frequentists generally accept the invariance principle without question. This is not the case for intelligent lifeforms such as Bayesians. The invariance property of the likeihood ratio is incompatible with the Bayesian habit of assigning a probability distribution to a parameter. 38
Page 1 and 2: Student Notes To Accompany MS4214:
Page 3 and 4: 4.8 Worked Problems . . . . . . . .
Page 5 and 6: Suppose that, in order to learn som
Page 7 and 8: Example 1.4 (Blood pressure). We wi
Page 9 and 10: Chapter 2 The Theory of Estimation
Page 11 and 12: Thus if we carried out an infinite
Page 13 and 14: Problem 2.1. Let X have a binomial
Page 15 and 16: Problem 2.6. Let X1, . . . , Xn be
Page 17 and 18: Theorem 2.8 (Cramér Rao lower boun
Page 19 and 20: Let us propose ˆµ = ¯ X as an es
Page 21 and 22: Example 2.2 (Bernoulli Trials). Con
Page 23 and 24: Relative Likelihood 0.0 0.2 0.4 0.6
Page 25 and 26: Example 2.7 (Exponential distributi
Page 27 and 28: ℓ(µ) = −m ln µ − 1 m� ti
Page 29 and 30: 2.5 Multi-parameter Estimation Supp
Page 31 and 32: Lemma 2.9 (Joint distribution of th
Page 33 and 34: So, if | ˆ θ − θ0| is small, w
Page 35 and 36: Then the log-likelihood function is
Page 37: A regression of the empirical distr
Page 41 and 42: Event Probability Set 0 0 0 (1 −
Page 43 and 44: 2.10 Worked Problems The Problems 1
Page 45 and 46: (a) Show that ˆµ is an unbiased e
Page 47 and 48: 4. E(X2 ) = � ∞ 0 x2f(x)dx =
Page 49 and 50: Student Questions 1. Let X1, X2, .
Page 51 and 52: Chapter 3 The Theory of Confidence
Page 53 and 54: Suppose that we have data X1, X2, .
Page 55 and 56: Lemma 3.2 (The Student t-distributi
Page 57 and 58: Example 3.6. Suppose that we have d
Page 59 and 60: 3.3 Approximate Confidence Interval
Page 61 and 62: 3.4 Worked Problems The Problems 1.
Page 63 and 64: Student Questions 1. Let X1, X2, .
Page 65 and 66: Chapter 4 The Theory of Hypothesis
Page 67 and 68: Example 4.3 (The power function). S
Page 69 and 70: (d) Suppose Θ0 = {(µ, σ) : −
Page 71 and 72: (c) Θ0 = {(µ, µ,σ) : −∞ <
Page 73 and 74: The Score Test Statistic: This test
Page 75 and 76: 4.5 The Neyman-Pearson Lemma Suppos
Page 77 and 78: According to the Neyman-Pearson lem
Page 79 and 80: would consider to be an unusually l
Page 81 and 82: pendent we would expect the proport
Page 83 and 84: 3. Explain what is meant by the pow
Page 85 and 86: For the null hypothesis θ = 1, the
Page 87 and 88: 0.1820. Similarly P (X = 3) = 0.218
Page 89 and 90:
Appendix A Review of Probability A.
Page 91 and 92:
where Xi ∼ Bernoulli(θ) for i =
Page 93 and 94:
The expected value of the jth momen
Page 95 and 96:
A.3 Continuous Random Variables A.3
Page 97 and 98:
A.3.4 Gaussian Distribution A rando
Page 99 and 100:
A.3.5 Weibull Distribution The Weib
Page 101 and 102:
Next, let g(y) be a monotone decrea
Page 103 and 104:
That is, X + Y ∼ Pois(θ + λ). =
Page 105 and 106:
since � ∞ −∞ e−αu2 du =
Page 107 and 108:
A.4.4 The Bivariate Normal Distribu
Page 109 and 110:
A.5 Generating Functions Denote the
Page 111 and 112:
Density p.g.f. m.g.f. ch.f. c.g.f B
Page 113:
Uniform U(a, b) Continuous Distribu
show all

Student Notes To Accompany MS4214: STATISTICAL INFERENCE

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?