Foundations of Data Science

Proof: Let
A =
r∑
σ i u i vi
T
i=1
be the SVD of A. If the rank of A is less than d, then for convenience complete
{v 1 , v 2 , . . . v r } into an orthonormal basis {v 1 , v 2 , . . . v d } of d-space. Write x in the basis
of the v i ’s as
d∑
x = c i v i .
Since (A T A) k = ∑ d
|c 1 | ≥ δ.
i=1 σ2k i
i=1
v i v T i , it follows that (AT A) k x = ∑ d
i=1 σ2k i
c i v i . By hypothesis,
Suppose that σ 1 , σ 2 , . . . , σ m are the singular values of A that are greater than or equal
to (1 − ε) σ 1 and that σ m+1 , . . . , σ d are the singular values that are less than (1 − ε) σ 1 .
Now
∣
|(A T A) k x| 2 d∑
=
∣
i=1
σ 2k
i c i v i
∣ ∣∣∣∣
2
=
d∑
i=1
σi
4k c 2 i ≥ σ1 4k c 2 1 ≥ σ1 4k δ 2 .
The component of |(A T A) k x| 2 perpendicular to the space V is
d∑
i=m+1
σi
4k c 2 i ≤ (1 − ε) 4k σ1
4k
d∑
i=m+1
c 2 i ≤ (1 − ε) 4k σ 4k
1
since ∑ d
i=1 c2 i = |x| = 1. Thus, the component of w perpendicular to V has squared
length at most (1−ε)4k σ1
4k
and so its length is at most
σ1 4kδ2
(1 − ε) 2k σ 2k
1
δσ 2k
1
=
(1 − ε)2k
δ
≤ e−2kε
δ
= ε.
Lemma 3.12 Let y ∈ R n be a random vector with the unit variance spherical Gaussian
as its probability density. Let x = y/|y|. Let v be any fixed (not random) unit length
vector. Then
(
Prob |x T v| ≤ 1 )
20 √ ≤ 1
d 10 + 3e−d/64 .
Proof: With c = √ d substituted in Theorem (2.9) of Chapter 2, the probability that
|y| ≥ 2 √ d is at most 3e −d/64 . Further, y T v is a random, zero mean, unit variance
Gaussian. Thus, the probability that |y T v| ≤ 1 is at most 1/10. Combining these two
10
facts and using the union bound, establishes the lemma.
51