Foundations of Data Science

Lemma 12.23 Let A be a symmetric matrix. Then ‖A‖ 2
= max ∣ x T Ax ∣ .
|x|=1
Proof: By definition, the 2-norm of A is ‖A‖ 2
= max |Ax|. Thus,
|x|=1
‖A‖ 2
= max
|x|=1
|Ax| = max
|x|=1
√
xT A T Ax = √ λ 2 1 = λ 1 = max
|x|=1
∣ x T Ax ∣ The two norm of a matrix A is greater than or equal to the 2-norm of any of its
columns. Let a u be a column of A.
Lemma 12.24 |a u | ≤ ‖A‖ 2
Proof: Let e u be the unit vector with a 1 in position u and all other entries zero. Note
λ = max |Ax|. Let x = e u where a u is row u. Then |a u | = |Ae u | ≤ max |Ax| = λ
|x|=1 |x|=1
12.7.7 Linear Algebra
Lemma 12.25 Let A be an n × n symmetric matrix. Then det(A) = λ 1 λ 2 · · · λ n .
Proof: The det (A − λI) is a polynomial in λ of degree n. The coefficient of λ n will be ±1
depending on whether n is odd or even. Let the roots of this polynomial be λ 1 , λ 2 , . . . , λ n .
Then det(A − λI) = (−1) n
n ∏
i=1
(λ − λ i ). Thus
det(A) = det(A − λI)| λ=0
= (−1) n
n
∏
i=1
(λ − λ i )
= λ 1 λ 2 · · · λ n
∣
λ=0
The trace of a matrix is defined to be the sum of its diagonal elements.
tr (A) = a 11 + a 22 + · · · + a nn .
That is,
Lemma 12.26 tr(A) = λ 1 + λ 2 + · · · + λ n .
Proof: Consider the coefficient of λ n−1 in det(A − λI) = (−1) n
A − λI =
⎛
⎜
⎝
a 11 − λ a 12 · · ·
a 21 a 22 − λ · · ·
.
.
.
⎞
⎟
⎠ .
n ∏
i=1
(λ − λ i ). Write
Calculate det(A − λI) by expanding along the first row. Each term in the expansion
involves a determinant of size n − 1 which is a polynomial in λ of deg n − 2 except for
the principal minor which is of deg n − 1. Thus the term of deg n − 1 comes from
(a 11 − λ) (a 22 − λ) · · · (a nn − λ)
415