Foundations of Data Science

Exercise 3.18 Modify the power method to find the first four singular vectors of a matrix
A as follows. Randomly select four vectors and find an orthonormal basis for the space
spanned by the four vectors. Then multiple each of the basis vectors times A and find a
new orthonormal basis for the space spanned by the resulting four vectors. Apply your
method to find the first four singular vectors of matrix A of Exercise 3.17. In Matlab the
command orth finds an orthonormal basis for the space spanned by a set of vectors.
Exercise 3.19 A matrix M is positive semi-definite if for all x, x T Mx ≥ 0.
1. Let A be a real valued matrix. Prove that B = AA T is positive semi-definite.
2. Let A be the adjacency matrix of a graph. The Laplacian of A is L = D − A where
D is a diagonal matrix whose diagonal entries are the row sums of A. Prove that
L is positive semi definite by showing that L = B T B where B is an m-by-n matrix
with a row for each edge in the graph, a column for each vertex, and we define
⎧
⎨
b ei =
⎩
−1 if i is the endpoint of e with lesser index
1 if i is the endpoint of e with greater index
0 if i is not an endpoint of e
Exercise 3.20 Prove that the eigenvalues of a symmetric real valued matrix are real.
Exercise 3.21 Suppose A is a square invertible matrix and the SVD of A is A = ∑ i
Prove that the inverse of A is ∑ i
1
σ i
v i u T i .
σ i u i v T i .
i=1
i=1
Exercise 3.22 Suppose A is square, but not necessarily invertible and has SVD A =
r∑
∑
σ i u i vi T . Let B =
r 1
σ i
v i u T i . Show that BAx = x for all x in the span of the rightsingular
vectors of A. For this reason B is sometimes called the pseudo inverse of A and
can play the role of A −1 in many applications.
Exercise 3.23
1. For any matrix A, show that σ k ≤ ||A|| F √
k
.
2. Prove that there exists a matrix B of rank at most k such that ||A − B|| 2 ≤ ||A|| F √
k
.
3. Can the 2-norm on the left hand side in (b) be replaced by Frobenius norm?
Exercise 3.24 Suppose an n × d matrix A is given and you are allowed to preprocess
A. Then you are given a number of d-dimensional vectors x 1 , x 2 , . . . , x m and for each of
these vectors you must find the vector Ax j approximately, in the sense that you must find a
vector y j satisfying |y j − Ax j | ≤ ε||A|| F |x j |. Here ε >0 is a given error bound. Describe
an algorithm that accomplishes this in time O ( )
d+n
ε per 2 xj not counting the preprocessing
time. Hint: use Exercise 3.23.
67