PDF of Lecture Notes - School of Mathematical Sciences

1. DISTRIBUTION THEORY
Theorem. 1.11.1 (Weak law of large numbers)
Suppose X 1 , X 2 , X 3 , . . . is a sequence of i.i.d. RVs with E[X i ] = µ, Var(X i ) = σ 2 , and
let
¯X n = 1 n∑
X i .
n
Then ¯X n converges to µ in probability.
Proof. We need to show for each ɛ > 0 that
i=1
lim P (| ¯X n − µ| > ɛ) = 0.
n→∞
Now observe that E[ ¯X n ] = µ and Var( ¯X n ) = σ2
. So by Chebyshev’s inequality,
n
P (| ¯X n − µ| > ɛ) ≤ σ2
→ 0 as n → ∞, for any fixed ɛ > 0.
nɛ2 Remarks
1. The proof given for Theorem 1.11.1 is really a corollary to the fact that ¯X n also
converges to µ in quadratic mean.
2. There is also a version of this theorem involving almost sure convergence (strong
law of large numbers). We will not discuss this.
3. The law of large numbers is one of the fundamental principles of statistical inference.
That is, it is the formal justification for the claim that the “sample mean
approaches the population mean for large n”.
Lemma. 1.11.1
Suppose a n is a sequence of real numbers s.t. lim
n→∞
na n = a with |a| < ∞. Then,
Proof.
Omitted (but not difficult).
lim (1 + a n) n = e a .
n→∞
73