v2006.03.09 - Convex Optimization

2.2. VECTORIZED-MATRIX INNER PRODUCT 572.2 Vectorized-matrix inner productEuclidean space R n comes equipped with a linear vector inner-product〈y,z〉 ∆ = y T z (25)Two vectors are orthogonal (perpendicular) to one another if and only if theirinner product vanishes;A vector inner-product defines a normy ⊥ z ⇔ 〈y,z〉 = 0 (26)‖y‖ 2 ∆ = √ y T y , ‖y‖ 2 = 0 ⇔ y = 0 (27)When orthogonal vectors each have unit norm, then they are orthonormal.For linear operation A on a vector, represented by a real matrix, the adjointoperation A T is transposition and defined for matrix A by [140,3.10]〈y,A T z〉 ∆ = 〈Ay,z〉 (28)The vector inner-product for matrices is calculated just as it is for vectors;by first transforming a matrix in R p×k to a vector in R pk by concatenatingits columns in the natural order. For lack of a better term, we shall callthat linear bijective (one-to-one and onto [140, App.A1.2]) transformationvectorization. For example, the vectorization of Y = [y 1 y 2 · · · y k ] ∈ R p×k[94] [208] is⎡ ⎤y 1vec Y =∆ y⎢ 2⎥⎣ . ⎦ ∈ Rpk (29)y kThen the vectorized-matrix inner-product is trace of matrix inner-product;for Z ∈ R p×k , [38,2.6.1] [123,0.3.1] [251,8] [238,2.2]where (A.1.1)〈Y , Z〉 ∆ = tr(Y T Z) = vec(Y ) T vec Z (30)tr(Y T Z) = tr(ZY T ) = tr(YZ T ) = tr(Z T Y ) = 1 T (Y ◦ Z)1 (31)