v2007.11.26 - Convex Optimization

4.5. CONSTRAINING CARDINALITY 275This sequence is iterated until x ⋆T y ⋆ vanishes; id est, until desired cardinalityis achieved.Vector y may be interpreted as a negative search direction; it pointsopposite to direction of movement of hyperplane {x | 〈x , y〉=τ} in aminimization of real linear function 〈x , y〉 over the feasible set in linearprogram (649). (p.64) Direction vector y is not unique. The feasible setof direction vectors in (435) is the convex hull of all cardinality-(n−k)one-vectors; videlicet,conv{u∈ R n | cardu = n−k , u i ∈ {0, 1}} = {a∈ R n | 1 ≽ a ≽ 0, 〈1, a〉= n−k}(650)Set {u∈ R n | cardu = n−k , u i ∈ {0, 1}} comprises the extreme points ofset (650); a hypercube slice. An optimal solution y to (435), that is anextreme point, is known in closed form; it has 1 in each entry correspondingto the n−k smallest entries of x ⋆ . That particular direction (−y) can beinterpreted as pointing toward cardinality-k vectors having the same orderingas x ⋆ . When y = 1, as in 1-norm minimization for example, then −y pointsdirectly at the origin (the cardinality-0 vector).This technique for constraining cardinality works often and, for someproblem classes (beyond Ax = b), it works all the time; meaning, optimalsolution to problem (647) can often be found by this convex iteration. Butexamples can be found that make the iteration stall at a solution not ofdesired cardinality. Heuristics for breaking out of a stall can be implementedwith some success:4.5.1.0.1 Example. Sparsest solution to Ax = b.Given data, from Example 4.2.3.1.1,⎡⎤ ⎡−1 1 8 1 1 0⎢1 1 1A = ⎣ −3 2 8 − 1 ⎥ ⎢2 3 2 3 ⎦ , b = ⎣−9 4 8141914 − 1 911214⎤⎥⎦ (579)the sparsest solution to the classical linear equation Ax = b is x = e 4 ∈ R 6(confer (592)). And given data, from Example 4.2.3.1.2,[ ] [ ] [ ]1 0 √2 12 1 11A =1or, b = (596)0 1 √2 1 1 21