v2007.11.26 - Convex Optimization

202 CHAPTER 3. GEOMETRY OF CONVEX FUNCTIONSProof. Optimal vector x ⋆ is orthogonal to the last n −1 columns oforthogonal matrix Q , sof ⋆ (478) = ‖x⋆ ‖ 2 (1 − (1 + ǫ) −1 ) (481)after each iteration. Convergence of f ⋆ is proven with the observation(478)that iteration (478) (479a) is a nonincreasing sequence that is bounded belowby 0. Any bounded monotonic sequence in R is convergent. [190,1.2][30,1.1] Expression (480) for optimal projector W holds at each iteration,therefore ‖x ⋆ ‖ 2 (1 − (1 + ǫ) −1 ) must also represent the optimal objectivevalue f ⋆ at convergence.(478)Because the objective f (477) from problem (477) is also bounded belowby 0 on the same domain, this convergent optimal objective value f ⋆ (for (478)positive ǫ arbitrarily close to 0) is necessarily optimal for (477); id est,by (1488), andf ⋆ (478) ≥ f⋆ (477) ≥ 0 (482)lim = 0 (483)ǫ→0 +f⋆ (478)Since optimal (x ⋆ , U ⋆ ) from problem (478) is feasible to problem (477), andbecause their objectives are equivalent for projectors by (474), then converged(x ⋆ , U ⋆ ) must also be optimal to (477) in the limit. Because problem (477)is convex, this represents a globally optimal solution.3.1.7.2 Semidefinite program via SchurSchur complement (1337) can be used to convert a projection problemto an optimization problem in epigraph form. Suppose, for example,we are presented with the constrained projection problem studied byHayden & Wells in [133] (who provide analytical solution): Given A∈ R M×Mand some full-rank matrix S ∈ R M×L with L < Mminimize ‖A − X‖ 2X∈ S MFsubject to S T XS ≽ 0(484)Variable X is constrained to be positive semidefinite, but only on a subspacedetermined by S . First we write the epigraph form: