Foundations of Data Science

⎡
⎢
⎣
A
n × m
⎤
⎡
≈
⎥ ⎢
⎦ ⎣
Sample
columns
n × s
⎤
⎥
⎦
[ Multi
plier
s × r
][ Sample rows
r × m
]
Figure 7.4: Schematic diagram of the approximation of A by a sample of s columns and
r rows.
uct j. The objective is to collect a few sample entries of A and based on them, get an
approximation to A so that we can make future recommendations. A few sampled rows
of A ( preferences of a few customers) and a few sampled columns (customers’ preferences
for a few products) give a good approximation to A provided that the samples are drawn
according to the length-squared distribution.
It remains now to describe how to find U from C and R. There is a n × n matrix P
of the form P = QR that acts as the identity on the space spanned by the rows of R and
zeros out all vectors orthogonal to this space. We state this now and postpone the proof.
Lemma 7.6 If RR T is invertible, then P = R T (RR T ) −1 R has the following properties:
(i) It acts as the identity matrix on the row space of R. I.e., P x = x for every vector x
of the form x = R T y (this defines the row space of R). Furthermore,
(ii) if x is orthogonal to the row space of R, then P x = 0.
If RR T is not invertible, let rank (RR T ) = r and RR T = ∑ r
t=1 σ tu t v T t be the SVD of
RR T . Then,
( r∑
)
P = R T 1 T
u
σ 2 t v t R
t=1 t
satisfies (i) and (ii).
We begin with some intuition. In particular, we first present a simpler idea that does
not work, but that motivates an idea that does. Write A as AI, where I is the n × n
identity matrix. Approximate the product AI using the algorithm of Theorem 7.5, i.e.,
by sampling s columns of A according to length-squared. Then, as in the last section,
write AI ≈ CW , where W consists of a scaled version of the s rows of I corresponding
to the s columns of A that were picked. Theorem 7.5 bounds the error ||A − CW || 2 F by
||A|| 2 F ||I||2 F /s = n s ||A||2 F . But we would like the error to be a small fraction of ||A||2 F which
254