Foundations of Data Science

Figure 8.4: Example of a bipartite graph.
We want a subset S of row vertices and a subset T of column vertices so that
A(S, T ) = ∑
i∈S,j∈T
is high. This simple definition is not good since A(S, T ) will be maximized by taking
all rows and columns. We need a balancing criterion that ensures that A(S, T ) is high
relative to the sizes of S and T . One possibility is to maximize A(S,T ) . This is not a good
|S||T |
measure either, since it is maximized by the single edge of highest weight. The definition
we use is the following. Let A be a matrix with nonnegative entries. For a subset S of
rows and a subset T of columns, the density d(S, T ) of S and T is d(S, T ) = √ A(S,T ) . The
|S||T |
density d(A) of A is defined as the maximum value of d(S, T ) over all subsets of rows and
columns. This definition applies to bipartite as well as non bipartite graphs.
One important case is when A’s rows and columns both represent the same set and
a ij is the similarity between object i and object j. Here d(S, S) = A(S,S) . If A is an n × n
|S|
0-1 matrix, it can be thought of as the adjacency matrix of an undirected graph, and
d(S, S) is the average degree of a vertex in S. The subgraph of maximum average degree
in a graph can be found exactly by network flow techniques, as we will show in the next
section. We do not know an efficient (polynomial-time) algorithm for finding d(A) exactly
in general. However, we show that d(A) is within a O(log 2 n) factor of the top singular
value of A assuming |a ij | ≤ 1 for all i and j. This is a theoretical result. The gap may be
much less than O(log 2 n) for many problems, making singular values and singular vectors
quite useful. Also, S and T with d(S, T ) ≥ Ω(d(A)/ log 2 n) can be found algorithmically.
Theorem 8.11 Let A be an n × d matrix with entries between 0 and 1. Then
σ 1 (A) ≥ d(A) ≥
Furthermore, subsets S and T satisfying d(S, T ) ≥
singular vector of A.
a ij
σ 1(A)
4 log n log d .
σ 1(A)
4 log n log d
may be found from the top
Proof: Let S and T be the subsets of rows and columns that achieve d(A) = d(S, T ).
1
Consider an n-vector u which is √
1
on S and 0 elsewhere and a d-vector v which is
|S|
on T and 0 elsewhere. Then,
285
√
|T |