13 - People.stat.sfu.ca - Simon Fraser University

Lecture 13: Method of Moments
To estimate p parameters solve
¯X = E θ (X)
X¯
2 = E θ (X 2 )
. .
X¯
p = E θ (X p )
Essentially always consistent.
Did Gamma(α,β) example.
Used as starting points for Newton-Raphson.
To solve g(θ) = 0 begin with initial value θ 0 and iteratively define
( −1g(θ
θ (k+1) = θ (k) − Dg(θ )) (k) (k) ).
Here Dg is the p ×p matrix with i,jth
∂g i (θ)
∂θ j
.
Richard Lockhart (Simon Fraser University) STAT 801=830 Statistical Inference Fall 2012 1 / 3

Lecture 13: quadratic approximation to l
Two term Taylor expansion of l about ˆθ:
[ ]
2 l(ˆθ)−l(θ) ≈ (θ− ˆθ) T I(θ 0 )(θ − ˆθ)
where minus the second derivative matrix has been replaced by the
Fisher information matrix.
Worked on large sample theory of above evaluated at θ = θ 0 .
Began with X ∼ MVN p (0,Σ) and showed
X T Σ −1 X ∼ χ 2 p.
Then started general theory of quadratic form:
X t BX = (AZ) T B(AZ) = Z T QZ
where Q = A T BA and AA T = Σ.
Write Q = PΛP T where P is orthogonal (PP T = P T P = I) and Λ is
diagonal. Columns of P are eigenvectors of Q and diagonal entries of
Λ are eigenvalues.
Richard Lockhart (Simon Fraser University) STAT 801=830 Statistical Inference Fall 2012 2 / 3

Coverage in the text
Method of moments in Chapter 9.
See “course notes” on web pages 130-131.
Richard Lockhart (Simon Fraser University) STAT 801=830 Statistical Inference Fall 2012 3 / 3