From Algorithms to Z-Scores - matloff - University of California, Davis

12.2. BIAS AND VARIANCE 247
people, and for W and s. For convenience, let’s suppose we record that last column as s 2 instead
of s.
Now, say we want to estimate the population variance σ 2 . As discussed earlier, the natural estimator
for it would be the sample variance, s 2 . What (12.39) says is that after looking at an infinite number
of lines in the notebook, the average value of s 2 would be just...a...little...bit...too...small. All the
s 2 values would average out to 0.999σ 2 , rather than to σ 2 . We might say that s 2 has a little bit
more tendency to underestimate σ 2 than to overestimate it.
So, (12.39) implies that s 2 is a biased estimator of the population variance σ 2 , with the amount of
bias being
n − 1
n · σ2 − σ 2 = − 1
· σ2
n
(12.40)
Let’s prove (12.39). As before, let W be a random variable distributed as the population, and let
W1, ..., Wn be a random sample from that population. So, EWi = µ and V ar(Wi) − σ 2 , where
again µ and σ 2 are the population mean and variance.
It will be more convenient to work with ns 2 than s 2 , since it will avoid a lot of dividing by n. So,
write
ns 2 =
=
=
But that middle sum is
So,
n
(Wi − W ) 2
i=1
n
(Wi − µ) + (µ − W ) 2 i=1
n
(Wi − µ) 2 + 2(µ − W )
i=1
n
(Wi − µ) =
i=1
ns 2 =
(def.) (12.41)
(alg.) (12.42)
n
(Wi − µ) + n(µ − W ) 2
i=1
(alg.) (12.43)
n
Wi − nµ = nW − nµ (12.44)
i=1
n
(Wi − µ) 2 − n(W − µ) 2
i=1
(12.45)