Foundations of Data Science

∑
λ 1 ≥ λ i , 2 ≤ i ≤ n, y = (1, 0, . . . , 0) maximizes n λ i yi
2 at λ 1 . Then x = P y is the first
column of P and is the first eigenvector of A. Similarly λ n is the minimum value of x T Ax
subject to the same conditions.
Now consider maximizing x T Ax subject to the conditions
1. ∑ x 2 i = 1
2. r T i x = 0
where the r i are any set of nonzero vectors. We ask over all possible choices of s vectors
what is the minimum value assumed by this maximum.
min
r 1 ,...,r s
i=1
max x T Ax
x
r T i x=0
As above, we may work with y. The conditions are
1. ∑ y 2 i = 1
2. q T i y = 0 where, q T i = r T i P
Consider any choice for the vectors r 1 , r 2 , . . . , r s . This gives a corresponding set of q i . The
y i therefore satisfy s linear homogeneous equations. If we add y s+2 = y s+3 = · · · y n = 0
we have n − 1 homogeneous equations in n unknowns y 1 , . . . , y n . There is at least one
solution that can be normalized so that ∑ y 2 i = 1. With this choice of y
y T Dy = ∑ λ i y 2 i ≥λ s+1
since coefficients greater than or equal to s + 1 are zero. Thus, for any choice of r i there
will be a y such that
max (y T P T AP y) ≥ λ s+1
y
r T i y=0
and hence
min
r 1 ,r 2 ,...,r s
max (y T P T AP y) ≥ λ s+1 .
y
r T i y=0
However, there is a set of s constraints for which the minimum is less than or equal to
λ s+1 . Fix the relations to be y i = 0, 1 ≤ i ≤ s. There are s equations in n unknowns
and for any y subject to these relations
y T Dy =
n∑
λ i yi 2 ≤ λ s+1 .
s+1
Combining the two inequalities, min max y T Dy = λ s+1 .
410