Student Notes To Accompany MS4214: STATISTICAL INFERENCE

More documents

Recommendations

Info

Contents 1 Introduction 3 1.1 Motivating Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2 General Course Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2 The Theory of Estimation 8 2.1 The Frequentist Philosophy . . . . . . . . . . . . . . . . . . . . . . . . 8 2.2 The Frequentist Approach to Estimation . . . . . . . . . . . . . . . . . 10 2.3 Minimum-Variance Unbiased Estimation . . . . . . . . . . . . . . . . . 14 2.4 Maximum Likelihood Estimation . . . . . . . . . . . . . . . . . . . . . 18 2.5 Multi-parameter Estimation . . . . . . . . . . . . . . . . . . . . . . . . 28 2.6 Newton-Raphsom Optimization . . . . . . . . . . . . . . . . . . . . . . 31 2.7 The Invariance Principle . . . . . . . . . . . . . . . . . . . . . . . . . . 38 2.8 Optimality Properties of the MLE . . . . . . . . . . . . . . . . . . . . . 39 2.9 Data Reduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 2.10 Worked Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 3 The Theory of Confidence Intervals 50 3.1 Exact Confidence Intervals . . . . . . . . . . . . . . . . . . . . . . . . . 50 3.2 Pivotal Quantities for Use with Normal Data . . . . . . . . . . . . . . . 53 3.3 Approximate Confidence Intervals . . . . . . . . . . . . . . . . . . . . 58 3.4 Worked Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 4 The Theory of Hypothesis Testing 64 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 4.2 The General Testing Problem . . . . . . . . . . . . . . . . . . . . . . . 65 4.3 Hypothesis Testing for Normal Data . . . . . . . . . . . . . . . . . . . 66 4.4 Generally Applicable Test Procedures . . . . . . . . . . . . . . . . . . . 71 4.5 The Neyman-Pearson Lemma . . . . . . . . . . . . . . . . . . . . . . . 74 4.6 Goodness of Fit Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 4.7 The χ 2 Test for Contingency Tables . . . . . . . . . . . . . . . . . . . . 79 1
4.8 Worked Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 A Review of Probability 88 A.1 Expectation and Variance . . . . . . . . . . . . . . . . . . . . . . . . . 88 A.2 Discrete Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . 89 A.2.1 Bernoulli Distribution . . . . . . . . . . . . . . . . . . . . . . . 89 A.2.2 Binomial Distribution . . . . . . . . . . . . . . . . . . . . . . . 89 A.2.3 Geometric Distribution . . . . . . . . . . . . . . . . . . . . . . . 90 A.2.4 Negative Binomial Distribution . . . . . . . . . . . . . . . . . . 91 A.2.5 Hypergeometric Distribution . . . . . . . . . . . . . . . . . . . . 91 A.2.6 Poisson Distribution . . . . . . . . . . . . . . . . . . . . . . . . 92 A.2.7 Discrete Uniform Distribution . . . . . . . . . . . . . . . . . . . 93 A.2.8 The Multinomial Distribution . . . . . . . . . . . . . . . . . . . 93 A.3 Continuous Random Variables . . . . . . . . . . . . . . . . . . . . . . . 94 A.3.1 Uniform Distribution . . . . . . . . . . . . . . . . . . . . . . . . 94 A.3.2 Exponential Distribution . . . . . . . . . . . . . . . . . . . . . . 94 A.3.3 Gamma Distribution . . . . . . . . . . . . . . . . . . . . . . . . 95 A.3.4 Gaussian Distribution . . . . . . . . . . . . . . . . . . . . . . . 96 A.3.5 Weibull Distribution . . . . . . . . . . . . . . . . . . . . . . . . 98 A.3.6 Beta Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . 98 A.3.7 Chi-square Distribution . . . . . . . . . . . . . . . . . . . . . . 99 A.3.8 Distribution of a Function of a Random Variable . . . . . . . . 99 A.4 Random Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 A.4.1 Sums of Independent Random Variables . . . . . . . . . . . . . 101 A.4.2 Covariance and Correlation . . . . . . . . . . . . . . . . . . . . 104 A.4.3 The Bivariate Change of Variables Formula . . . . . . . . . . . 105 A.4.4 The Bivariate Normal Distribution . . . . . . . . . . . . . . . . 106 A.4.5 Bivariate Normal Conditional Distributions . . . . . . . . . . . 107 A.4.6 The Multivariate Normal Distribution . . . . . . . . . . . . . . 107 A.5 Generating Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 A.6 Table of Common Distributions . . . . . . . . . . . . . . . . . . . . . . 111 2
Page 1: Student Notes To Accompany MS4214:
Page 5 and 6: Suppose that, in order to learn som
Page 7 and 8: Example 1.4 (Blood pressure). We wi
Page 9 and 10: Chapter 2 The Theory of Estimation
Page 11 and 12: Thus if we carried out an infinite
Page 13 and 14: Problem 2.1. Let X have a binomial
Page 15 and 16: Problem 2.6. Let X1, . . . , Xn be
Page 17 and 18: Theorem 2.8 (Cramér Rao lower boun
Page 19 and 20: Let us propose ˆµ = ¯ X as an es
Page 21 and 22: Example 2.2 (Bernoulli Trials). Con
Page 23 and 24: Relative Likelihood 0.0 0.2 0.4 0.6
Page 25 and 26: Example 2.7 (Exponential distributi
Page 27 and 28: ℓ(µ) = −m ln µ − 1 m� ti
Page 29 and 30: 2.5 Multi-parameter Estimation Supp
Page 31 and 32: Lemma 2.9 (Joint distribution of th
Page 33 and 34: So, if | ˆ θ − θ0| is small, w
Page 35 and 36: Then the log-likelihood function is
Page 37 and 38: A regression of the empirical distr
Page 39 and 40: 2.7 The Invariance Principle How do
Page 41 and 42: Event Probability Set 0 0 0 (1 −
Page 43 and 44: 2.10 Worked Problems The Problems 1
Page 45 and 46: (a) Show that ˆµ is an unbiased e
Page 47 and 48: 4. E(X2 ) = � ∞ 0 x2f(x)dx =
Page 49 and 50: Student Questions 1. Let X1, X2, .
Page 51 and 52: Chapter 3 The Theory of Confidence
Page 53 and 54:
Suppose that we have data X1, X2, .
Page 55 and 56:
Lemma 3.2 (The Student t-distributi
Page 57 and 58:
Example 3.6. Suppose that we have d
Page 59 and 60:
3.3 Approximate Confidence Interval
Page 61 and 62:
3.4 Worked Problems The Problems 1.
Page 63 and 64:
Student Questions 1. Let X1, X2, .
Page 65 and 66:
Chapter 4 The Theory of Hypothesis
Page 67 and 68:
Example 4.3 (The power function). S
Page 69 and 70:
(d) Suppose Θ0 = {(µ, σ) : −
Page 71 and 72:
(c) Θ0 = {(µ, µ,σ) : −∞ <
Page 73 and 74:
The Score Test Statistic: This test
Page 75 and 76:
4.5 The Neyman-Pearson Lemma Suppos
Page 77 and 78:
According to the Neyman-Pearson lem
Page 79 and 80:
would consider to be an unusually l
Page 81 and 82:
pendent we would expect the proport
Page 83 and 84:
3. Explain what is meant by the pow
Page 85 and 86:
For the null hypothesis θ = 1, the
Page 87 and 88:
0.1820. Similarly P (X = 3) = 0.218
Page 89 and 90:
Appendix A Review of Probability A.
Page 91 and 92:
where Xi ∼ Bernoulli(θ) for i =
Page 93 and 94:
The expected value of the jth momen
Page 95 and 96:
A.3 Continuous Random Variables A.3
Page 97 and 98:
A.3.4 Gaussian Distribution A rando
Page 99 and 100:
A.3.5 Weibull Distribution The Weib
Page 101 and 102:
Next, let g(y) be a monotone decrea
Page 103 and 104:
That is, X + Y ∼ Pois(θ + λ). =
Page 105 and 106:
since � ∞ −∞ e−αu2 du =
Page 107 and 108:
A.4.4 The Bivariate Normal Distribu
Page 109 and 110:
A.5 Generating Functions Denote the
Page 111 and 112:
Density p.g.f. m.g.f. ch.f. c.g.f B
Page 113:
Uniform U(a, b) Continuous Distribu
show all

Student Notes To Accompany MS4214: STATISTICAL INFERENCE

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?