Foundations of Data Science

and has coefficient (−1) n−1 (a 11 + a 22 + · · · + a nn ). Now
(−1) n
n
∏
i=1
(λ − λ i ) = (−1) n (λ − λ 1 )(λ − λ 2 ) · · · (λ − λ n )
( )
= (−1) n λ n − (λ 1 + λ 2 + · · · + λ n )λ n−1 + · · ·
Therefore equating coefficients λ 1 + λ 2 + · · · + λ n = a 11 + a 22 + · · · + a nn = tr(A)
( )
( )
1 0
1 0
Note that (tr(A)) 2 ≠ tr(A 2 ). For example A = has trace 3, A
0 2
2 =
0 4
has trace 5 ≠9. However tr(A 2 ) = λ 2 1 + λ 2 2 + · · · + λ 2 n. To see this, observe that A 2 =
(V T DV ) 2 = V T D 2 V . Thus, the eigenvalues of A 2 are the squares of the eigenvalues for
A.
Alternative proof that tr(A) = λ 1 + λ 2 + · · ·+ λ n . Suppose the spectral decomposition
of A is A = P DP T . We have
tr (A) = tr ( P DP T ) = tr ( DP T P ) = tr (D) = λ 1 + λ 2 + · · · + λ n .
Lemma 12.27 If A is n × m and B is a m × n matrix, then tr(AB)=tr(BA).
Pseudo inverse
tr(AB) =
n∑ n∑
a ij b ji =
i=1 j=1
n∑ n∑
b ji a ij = tr (BA)
j=1 i=1
Let A be an n × m rank ( r matrix and let A ) = UΣV T be the singular value decomposition
of A. Let Σ ′ 1
= diag
σ 1
, . . . , 1 σ r
, 0, . . . , 0 where σ 1 , . . . , σ r are the nonzero singular
values of A. Then A ′ = V Σ ′ U T is the pseudo inverse of A. It is the unique X that
minimizes ‖AX − I‖ F
.
Second eigenvector
Suppose the eigenvalues of a matrix are λ 1 ≥ λ 2 ≥ · · · . The second eigenvalue,
λ 2 , plays an important role for matrices representing graphs. It may be the case that
|λ n | > |λ 2 |.
Why is the second eigenvalue so important? Consider partitioning the vertices of a
regular degree d graph G = (V, E) into two blocks of equal size so as to minimize the
number of edges between the two blocks. Assign value +1 to the vertices in one block and
-1 to the vertices in the other block. Let x be the vector whose components are the ±1
values assigned to the vertices. If two vertices, i and j, are in the same block, then x i and
x j are both +1 or both –1 and (x i −x j ) 2 = 0. If vertices i and j are in different blocks then
(x i − x j ) 2 = 4. Thus, partitioning the vertices into two blocks so as to minimize the edges
416