Foundations of Data Science

would require s ≥ n, which clearly is of no use since this would pick as many or more
columns than the whole of A.
Let’s use the identity-like matrix P instead of I in the above discussion. Using the
fact that R is picked according to length squared sampling, we will show the following
proposition later.
Proposition 7.7 A ≈ AP and the error E (||A − AP || 2 2) is at most
1 √r ||A|| 2 F .
We then use Theorem 7.5 to argue that instead of doing the multiplication AP , we can
use the sampled columns of A and the corresponding rows of P . The s sampled columns
of A form C. We have to take the corresponding s rows of P = R T (RR T ) −1 R, which is
the same as taking the corresponding s rows of R T , and multiplying this by (RR T ) −1 R. It
is easy to check that this leads to an expression of the form CUR. Further, by Theorem
7.5, the error is bounded by
E ( ) ( )
||AP − CUR|| 2 2 ≤ E ||AP − CUR||
2 ||A|| 2
F ≤
F ||P ||2 F
s
since we will show later that:
≤ r s ||A||2 F , (7.6)
Proposition 7.8 ||P || 2 F ≤ r.
Putting (7.6) and Proposition 7.7 together, and using the fact that by triangle inequality
||A − CUR|| 2 ≤ ||A − AP || 2 + ||AP − CUR|| 2 , which in turn implies that ||A − CUR|| 2 2 ≤
2||A − AP || 2 2 + 2||AP − CUR|| 2 2. The main result follows.
Theorem 7.9 Let A be an m × n matrix and r and s be positive integers. Let C be an
m × s matrix of s columns of A picked according to length squared sampling and let R be
a matrix of r rows of A picked according to length squared sampling. Then, we can find
from C and R an s × r matrix U so that
E ( (
)
||A − CUR|| 2 2 ≤ ||A||
2 2√r
F + 2r )
.
s
If s is fixed, the error is minimized when r = s 2/3 . Choosing s = r/ε and r = 1/ε 2 , the
bound becomes O(ε)||A|| 2 F . When is this bound meaningful? We discuss this further after
first proving all the claims used in the discussion above.
Proof: (of Lemma (7.6)): First for the case when RR T is invertible. For x = R T y,
R T (RR T ) −1 Rx = R T (RR T ) −1 RR T y = R T y = x. If x is orthogonal to every row of R,
then Rx = 0, so P x = 0. More generally, if RR T = ∑ t σ tu t v T t , then, R ∑ T 1
t
R =
σ
∑
t 2
t v tv T t and clearly satisfies (i) and (ii).
Next we prove Proposition 7.7. First, recall that
||A − AP || 2 2 =
max |(A − AP
{x:|x|=1} )x|2 .
255