Foundations of Data Science

Replacing e tx by its power series expansion 1 + tx + (tx)2 · · · gives
2!
∫ ∞ (
)
Ψ(t) = 1 + tx + (tx)2 + · · · p(x)dx
2!
−∞
Thus, the k th moment of x about the origin is k! times the coefficient of t k in the power
series expansion of the moment generating function. Hence, the moment generating function
is the exponential generating function for the sequence of moments about the origin.
The moment generating function transforms the probability distribution p(x) into a
function Ψ (t) of t. Note Ψ(0) = 1 and is the area or integral of p(x). The moment
generating function is closely related to the characteristic function which is obtained by
replacing e tx by e itx in the above integral where i = √ −1 and is related to the Fourier
transform which is obtained by replacing e tx by e −itx .
Ψ(t) is closely related to the Fourier transform and its properties are essentially the
same. In particular, p(x) can be uniquely recovered by an inverse transform from Ψ(t).
∑
More specifically, if all the moments m i are finite and the sum ∞ m i
i! ti converges absolutely
in a region around the origin, then p(x) is uniquely determined.
The Gaussian probability distribution with zero mean and unit variance is given by
p (x) = √ 1
2π
e − x2
2 . Its moments are given by
u n = 1 √
2π
=
∫∞
−∞
x n e − x2
2 dx
{ n!
n even
2 n 2 (
n
2 )!
0 n odd
i=0
To derive the above, use integration by parts to get u n = (n − 1) u n−2 and combine
this with u 0 = 1 and u 1 = 0. The steps are as follows. Let u = e − x2
2 and v = x n−1 . Then
u ′ = −xe − x2
2 and v ′ = (n − 1) x n−2 . Now uv = ∫ u ′ v+ ∫ uv ′ or
∫
e − x2
2 x n−1 =
x n e − x2
2 dx +
∫
(n − 1) x n−2 e − x2
2 dx.
From which
∫
x n e − x2
2 dx = (n − 1) ∫ x n−2 e − x2
2 dx − e − x2
2 x n−1
∞∫
∞∫
x n e − x2
2 dx = (n − 1) x n−2 e − x2
2 dx
−∞
−∞
422