Foundations of Data Science

Thus, u n = (n − 1) u n−2 .
The moment generating function is given by
∞∑ u n s n ∞∑ n! s n
g (s) = =
n! 2 n 2 n! n! = ∑ ∞
2
n=0
n=0
n even
i=0
s 2i ∞
2 i i! = ∑
For the general Gaussian, the moment generating function is
(
g (s) = e su+ σ
2
2
i=0
1
i!
( ) s
2 i
= e s2 2 .
Thus, given two independent Gaussians with mean u 1 and u 2 and variances σ 2 1 and σ 2 2,
the product of their moment generating functions is
)
s 2
e s(u 1+u 2 )+(σ 2 1 +σ2 2)s 2 ,
the moment generating function for a Gaussian with mean u 1 + u 2 and variance σ 2 1 + σ 2 2.
Thus, the convolution of two Gaussians is a Gaussian and the sum of two random variables
that are both Gaussian is a Gaussian random variable.
2
12.9 Miscellaneous
12.9.1 Lagrange multipliers
Lagrange multipliers are used to convert a constrained optimization problem into an unconstrained
optimization. Suppose we wished to maximize a function f(x) subject to a
constraint g(x) = c. The value of f(x) along the constraint g(x) = c might increase for
a while and then start to decrease. At the point where f(x) stops increasing and starts
to decrease, the contour line for f(x) is tangent to the curve of the constraint g(x) = c.
Stated another way the gradient of f(x) and the gradient of g(x) are parallel.
By introducing a new variable λ we can express the condition by ∇ x f = λ∇ x g and
g = c. These two conditions hold if and only if
∇ xλ
(
f (x) + λ (g (x) − c)
)
= 0
The partial with respect to λ establishes that g(x) = c. We have converted the constrained
optimization problem in x to an unconstrained problem with variables x and λ.
12.9.2 Finite Fields
For a prime p and integer n there is a unique finite field with p n elements. In Section
4.6 we used the field GF(2 n ), which consists of polynomials of degree less than n with
coefficients over the field GF(2). In GF(2 8 )
(x 7 + x 5 + x) + (x 6 + x 5 + x 4 ) = x 7 + x 6 = x 4 = x
423