Foundations of Data Science

Proof: There is a set of r−1 relations such that over all x satisfying the r−1 relationships
max(x T Ax) = α r .
And a set of s − 1 relations such that over all x satisfying the s − 1 relationships
max(x T Bx) = β s .
Consider x satisfying all these r + s − 2 relations. For any such x
x T Cx = x T Ax + x T Bxx ≤ α r + β s
and hence over all the x
max(x T Cx) ≤ α s + β r
Taking the minimum over all sets of r + s − 2 relations
γ r+s−1 = min max(x T Cx) ≤ α r + β s
12.7.5 Norms
A set of vectors {x 1 , . . . , x n } is orthogonal if x T i x j = 0 for i ≠ j and is orthonormal if
in addition |x i | = 1 for all i. A matrix A is orthonormal if A T A = I. If A is a square
orthonormal matrix, then rows as well as columns are orthogonal. In other words, if A
is square orthonormal, then A T is also. In the case of matrices over the complexes, the
concept of an orthonormal matrix is replaced by that of a unitary matrix. A ∗ is the conjugate
transpose of A if a ∗ ij = ā ji where a ∗ ij is the ij th entry of A ∗ and ā ∗ ij is the complex
conjugate of the ij th element of A. A matrix A over the field of complex numbers is
unitary if AA ∗ = I.
Norms
A norm on R n is a function f : R n → R satisfying the following three axioms:
1. f(x) ≥ 0,
2. f(x + y) ≤ f(x) + f(y), and
3. f(αx) = |α|f(x).
A norm on a vector space provides a distance function where
distance(x, y) = norm(x − y).
An important class of norms for vectors is the p-norms defined for p > 0 by
|x| p
= (|x 1 | p + · · · + |x n | p ) 1 p .
412