v2007.09.17 - Convex Optimization

592 APPENDIX E. PROJECTIONE.3.2Orthogonal characterizationAny symmetric projector P 2 =P ∈ S m projecting on nontrivial R(Q) canbe defined by the orthonormality condition Q T Q = I . When skinny matrixQ∈ R m×k has orthonormal columns, then Q † = Q T by the Moore-Penroseconditions. Hence, any P having an orthonormal decomposition (E.3.4)where [249,3.3] (1392)P = QQ T , Q T Q = I (1680)R(P ) = R(Q) , N(P ) = N(Q T ) (1681)is an orthogonal projector projecting on R(Q) having, for Px∈ R(Q)(confer (1666))Px − x ⊥ R(Q) in R m (1682)From (1680), orthogonal projector P is obviously positive semidefinite(A.3.1.0.6); necessarily,P T = P , P † = P , ‖P ‖ 2 = 1, P ≽ 0 (1683)and ‖Px‖ = ‖QQ T x‖ = ‖Q T x‖ because ‖Qy‖ = ‖y‖ ∀y ∈ R k . All orthogonalprojectors are therefore nonexpansive because√〈Px, x〉 = ‖Px‖ = ‖Q T x‖ ≤ ‖x‖ ∀x∈ R m (1684)the Bessel inequality, [73] [166] with equality when x∈ R(Q).From the diagonalization of idempotent matrices (1660) on page 586P = SΦS T =m∑φ i s i s T i =i=1k∑≤ mi=1s i s T i (1685)orthogonal projection of point x on R(P ) has expression like an orthogonalexpansion [73,4.10]k∑Px = QQ T x = s T i xs i (1686)wherei=1Q = S(:,1:k) = [ s 1 · · · s k]∈ Rm×k(1687)and where the s i [sic] are orthonormal eigenvectors of symmetricidempotent P . When the domain is restricted to the range of P , sayx=Qξ for ξ ∈ R k , then x = Px = QQ T Qξ = Qξ and expansion is unique.Otherwise, any component of x in N(Q T ) will be annihilated.