Foundations of Data Science

establishing the first inequality.
σ 1 (A) ≥ u T Av = ∑ u i v j a ij = d(S, T ) = d(A)
ij
To prove the second inequality, express σ 1 (A) in terms of the first left and right
singular vectors x and y.
σ 1 (A) = x T Ay = ∑ i,j
x i a ij y j , |x| = |y| = 1.
Since the entries of A are nonnegative, the components of the first left and right singular
vectors
∑
must all be nonnegative, that is, x i ≥ 0 and y j ≥ 0 for all i and j. To bound
x i a ij y j , break the summation into O (log n log d) parts. Each part corresponds to a
i,j
given α and β and consists of all i such that α ≤ x i < 2α and all j such that β ≤ y i < 2β.
The log n log d parts are defined by breaking the rows into log n blocks with α equal to
1 √1
1
2 n
, √n , 2 √ 1
n
, 4 √ 1
n
, . . . , 1 and by breaking the columns into log d blocks with β equal to
1 √1
1
2 d
, √d 2
, √d 4
, √d , . . . , 1. The i such that x i < 1
2 √ and the j such that y n j < 1
2 √ will be
d
ignored at a loss of at most 1σ 4 1(A). Exercise (8.28) proves the loss is at most this amount.
Since ∑ x 2 i = 1, the set S = {i|α ≤ x i < 2α} has |S| ≤ 1 and similarly,
α 2
i
T = {j|β ≤ y j ≤ 2β} has |T | ≤ 1 . Thus
β 2
∑
i
α≤x i ≤2α
∑
x i y j a ij ≤ 4αβA(S, T )
j
β≤y j ≤2β
≤ 4αβd(S, T ) √ |S||T |
≤ 4d(S, T )
≤ 4d(A).
From this it follows that
or
proving the second inequality.
σ 1 (A) ≤ 4d (A) log n log d
d (A) ≥
σ 1(A)
4 log n log d
It is clear that for each of the values of (α, β), we can compute A(S, T ) and d(S, T )
as above and taking the best of these d(S, T ) ’s gives us an algorithm as claimed in the
theorem.
Note that in many cases, the nonzero values of x i and y j after zeroing out the low
entries will only go from 1 √1
2 n
to √ c
n
for x i and 1 √1
2 d
to √ c
d
for y j , since the singular vectors
286