v2004.06.19 - Convex Optimization

2.1. CONVEX SET 252.1.1 Vectorized matrix inner productEuclidean space R n comes equipped with a linear vector inner product〈y,z〉 ∆ = y T z (14)Two vectors are orthogonal (perpendicular) to one another if and only if theirinner product vanishes;y ⊥ z ⇔ 〈y,z〉 = 0 (15)An inner product defines a norm‖y‖ 2 ∆ = √ y T y , ‖y‖ 2 = 0 ⇔ y = 0 (16)When orthogonal vectors each have unit norm, then they are orthonormal.For linear operations on vectors represented by real matrices, the adjointoperation is transposition and defined for matrix operator A by [38, §3.10]〈Ay,z〉 = 〈y,A T z〉 (17)The vector inner product for matrices is calculated just as it is for vectorsby first transforming a matrix in R p×k to a vector in R pk by concatenatingits columns in the natural order. For lack of a better term, we shall call thatlinear bijective transformation vectorization. For example, the vectorizationof Y = [y 1 y 2 · · · y k ] ∈ R p×k [40] [41] isvec Y ∆ =⎡⎢⎣⎤y 1y 2⎥.y k⎦ ∈ Rpk (18)Then the vectorized-matrix inner product is the trace of the matrix innerproduct; for Z ∈ R p×k , [1, §2.6.1] [29, §0.3.1] [42, §8] [43, §2.2]where〈Y , Z〉 ∆ = tr(Y T Z) = vec(Y ) T vec Z (19)tr(Y T Z) = tr(ZY T ) = tr(YZ T ) = tr(Z T Y ) = 1 T (Y ◦ Z)1 (20)