Foundations of Data Science

Important special cases are
|x| 0 the number of non zero entries
|x| 1
= |x 1 | + · · · + |x n |
|x| 2
= √ |x 1 | 2 + · · · + |x n | 2
|x| ∞
= max |x i |.
Lemma 12.15 For any 1 ≤ p < q, |x| q ≤ |x| p .
Proof:
|x| q q = ∑ i
|x i | q .
Let a i = |x i | q and ρ = p/q. Using Jensen’s inequality (see Section 12.3) that for any
nonnegative reals a 1 , a 2 , . . . , a n and any ρ ∈ (0, 1), we have ( ∑ n
i=1 a i) ρ ≤ ∑ n
i=1 aρ i , the
lemma is proved.
There are two important matrix norms, the matrix p-norm
||A|| p
= max
|x|=1 ‖Ax‖ p
and the Frobenius norm
√ ∑
||A|| F
=
ij
a 2 ij .
Let a i be the i th column of A. Then ‖A‖ 2 F = ∑ i
a i T a i = tr ( A T A ) . A similar argument
on the rows yields ‖A‖ 2 F = tr ( AA T ) . Thus, ‖A‖ 2 F = tr ( A T A ) = tr ( AA T ) .
If A is symmetric and rank k
||A|| 2 2 ≤ ||A||2 F ≤ k ||A||2 2 .
12.7.6 Important Norms and Their Properties
Lemma 12.16 ||AB|| 2
≤ ||A|| 2
||B|| 2
Proof: ||AB|| 2
= max |ABx|. Let y be the value of x that achieves the maximum and
|x|=1
let z = By. Then
||AB|| 2
= |ABy| = |Az| = ∣ ∣
But
∣
∣A z
|z|
∣ ≤ max |Ax| = ||A|| 2
|x|=1
∣ A z
|z| ∣ |z|
and |z| ≤ max
|x|=1 |Bx| = ||B|| 2 . Thus ||AB|| 2 ≤ ||A|| 2 ||B|| 2 .
413