Stat 5101 Lecture Notes - School of Statistics

40 Stat 5101 (Geyer) Course Notes2.4.2 Second Moments and VariancesThe preceding section says all that can be said in general about first moments.As we shall now see, second moments are much more complicated.The most important second moment is the second central moment, whichalso has a special name. It is called the variance and is often denoted σ 2 . (Thesymbol σ is a Greek letter. See Appendix A). We will see the reason for thesquare presently. We also use the notation var(X) for the variance of X. Soσ 2 =µ 2 = var(X) =E{(X−µ) 2 }.As we did with means, when there are several random variables under discussion,we denote the variance of each using the same Greek letter σ, but add thevariable as a subscript to distinguish them: σX 2 = var(X), σ2 Y = var(Y ), and soforth.Note that variance is just an expectation like any other, the expectation ofthe random variable (X − µ) 2 .All second moments are related.Theorem 2.11 (Parallel Axis Theorem). If X is a random variable withmean µ and variance σ 2 , thenE{(X − a) 2 } = σ 2 +(µ−a) 2Proof. Using the factfrom algebra(b + c) 2 = b 2 +2bc + c 2 (2.14)(X − a) 2 =(X−µ+µ−a) 2=(X−µ) 2 +2(X−µ)(µ − a)+(µ−a) 2Taking expectations of both sides and applying linearity of expectation (everythingnot containing X is nonrandom and so can be pulled out of expectations)givesE{(X − a) 2 } = E{(X − µ) 2 } +2(µ−a)E(X−µ)+(µ−a) 2 E(1)= σ 2 +2(µ−a)µ 1 +(µ−a) 2By Theorem 2.9, the middle term on the right hand side is zero, and thatcompletes the proof.The name of this theorem is rather strange. It is taken from an analogoustheorem in physics about moments of inertia. So the name has nothing to dowith probability in general and moments (as understood in probability theoryrather than physics) in particular, and the theorem is not commonly calledby that name. We will use it because Lindgren does, and perhaps becausethe theorem doesn’t have any other widely used name. In fact, since it is so