Chapter 3 Quadratic Programming

Optimization I; Chapter 3 59
3.3.3 Null-space approach
The null-space approach does not require regularity of B and thus has a wider
range of applicability than the range-space approach.
We assume that A ∈ lR m×n has full row rank m and that Z T BZ is positive
definite, where Z ∈ lR n×(n−m) is the matrix whose columns span Ker A which
can be computed by QR factorization (cf. Chapter 2.4).
We partition the vector x ∗ according to
x ∗ = Y w Y + Zw Z , (3.7)
where Y ∈ lR n×m is such that [Y Z] ∈ lR n×n is nonsingular and w Y ∈ lR m , w Z ∈
lR n−m .
Substituting (3.7) into the second equation of (3.3), we obtain
Ax ∗ = AY w Y + }{{} AZ w Z = c , (3.8)
= 0
i.e., Y w Y is a particular solution of Ax = c.
Since A ∈ lR m×n has rank m and [Y Z] ∈ lR n×n is nonsingular, the product
matrix A[Y Z] = [AY 0] ∈ lR m×m is nonsingular. Hence, w Y is well determined
by (3.8).
On the other hand, substituting (3.7) into the first equation of (3.3), we get
BY w Y + BZw Z + A T λ ∗ = b .
Multiplying by Z T and observing Z T A T = (AZ) T = 0 yields
Z T BZw Z = Z T b − Z T BY w Y . (3.9)
The reduced KKT system (3.9) can be solved by a Cholesky factorization of the
reduced Hessian Z T BZ ∈ lR (n−m)×(n−m) . Once w Y and w Z have been computed
as the solutions of (3.8) and (3.9), x ∗ is obtained according to (3.7).
Finally, the Lagrange multiplier turns out to be the solution of the linear system
arising from the multiplication of the first equation in (3.7) by Y T :
(AY ) T λ ∗ = Y T b − Y T Bx ∗ . (3.10)
3.4 Iterative solution of the KKT system
If the direct solution of the KKT system (3.3) is computationally too costly,
the alternative is to use an iterative method. An iterative solver can be applied
either to the entire KKT system or, as in the range-space and null-space
approach, use the special structure of the KKT matrix.