v2006.03.09 - Convex Optimization

7.2. SECOND PREVALENT PROBLEM: 4117.2.2.3 Rank-heuristic insightMinimization of affine dimension by use of this trace rank-heuristic (989)tends to find the list configuration of least energy; rather, it tends to optimizecompaction of the reconstruction by minimizing total distance. (485) It is bestused where some physical equilibrium implies such an energy minimization;e.g., [228,5].For this Problem 2, the trace rank-heuristic arose naturally in theobjective in terms of V . We observe: V (in contrast to VN T ) spreads energyover all available distances (B.4.2 no.20, no.22) although the rank functionitself is insensitive to the choice.7.2.2.4 Rank minimization heuristic beyond convex envelopeFazel, Hindi, and Boyd [74] propose a rank heuristic more potent than trace(988) for problems of rank minimization;rankY ← log det(Y +εI) (992)the concave surrogate function log det in place of quasiconcave rankY(2.9.2.5.2) when Y ∈ S n + is variable and where ε is a small positive constant.They propose minimization of the surrogate by substituting a sequencecomprising infima of a linearized surrogate about the current estimate Y i ;id est, from the first-order Taylor series expansion about Y i on some openinterval of ‖Y ‖ (D.1.6)log det(Y + εI) ≈ log det(Y i + εI) + tr ( (Y i + εI) −1 (Y − Y i ) ) (993)we make the surrogate sequence of infima over bounded convex feasible set Cwherearg inf rankY ← lim Y i+1 (994)Y ∈ C i→∞Y i+1 = arg infY ∈ C tr( (Y i + εI) −1 Y ) (995)Choosing Y 0 =I , the first step becomes equivalent to finding the infimum oftrY ; the trace rank-heuristic (988). The intuition underlying (995) is thenew term in the argument of trace; specifically, (Y i + εI) −1 weights Y so thatrelatively small eigenvalues of Y found by the infimum are made even smaller.