Semidefinite programming Feasibility and duality - Imre Polik
Semidefinite programming Feasibility and duality - Imre Polik
Semidefinite programming Feasibility and duality - Imre Polik
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
IE496/4<br />
<strong>Imre</strong> Pólik<br />
Outline<br />
<strong>Feasibility</strong><br />
Duality<br />
<strong>Semidefinite</strong> <strong>programming</strong><br />
<strong>Feasibility</strong> <strong>and</strong> <strong>duality</strong><br />
<strong>Imre</strong> Pólik, PhD<br />
Lehigh University<br />
Department of Industrial <strong>and</strong> Systems Engineering<br />
January 22, 2009
IE496/4<br />
<strong>Imre</strong> Pólik<br />
Outline<br />
Outline<br />
<strong>Feasibility</strong><br />
Duality<br />
1 <strong>Feasibility</strong><br />
Review of LP<br />
Weak <strong>and</strong> strong infeasibility<br />
Farkas lemma for SDP<br />
2 Duality<br />
Review of LP <strong>duality</strong><br />
SDP <strong>duality</strong>
IE496/4<br />
<strong>Imre</strong> Pólik<br />
Farkas lemma<br />
Outline<br />
<strong>Feasibility</strong><br />
Review of LP<br />
Weak <strong>and</strong> strong<br />
infeasibility<br />
Farkas lemma for SDP<br />
Duality<br />
min c T x<br />
max b T y<br />
Ax = b<br />
A T y + s = c<br />
x ≥ 0 s ≥ 0<br />
Theorem (Farkas lemma)<br />
The following two statements are equivalent:<br />
1 There is an x ∈ R n such that Ax = b <strong>and</strong> x ≥ 0.<br />
2 There is no y ∈ R m such that A T y ≤ 0 <strong>and</strong> b T y = 1.<br />
In other words, primal (dual) infeasibility is equivalent to the<br />
existence of a dual (primal) improving direction.
Outline<br />
IE496/4<br />
<strong>Imre</strong> Pólik<br />
<strong>Feasibility</strong><br />
Review of LP<br />
Weak <strong>and</strong> strong<br />
infeasibility<br />
Farkas lemma for SDP<br />
Duality<br />
Weak infeasibility for SDP<br />
Consider<br />
(<br />
0 1<br />
min<br />
1 0<br />
(<br />
1 0<br />
0 0<br />
) ( )<br />
x11 x<br />
•<br />
12<br />
x 21 x 22<br />
)<br />
•<br />
( )<br />
x11 x 12<br />
= 1<br />
x 21 x 22<br />
(<br />
1 0<br />
0 0<br />
max y 1<br />
)<br />
(<br />
0 1<br />
y 1 + S =<br />
1 0<br />
X ≽ 0 S ≽ 0<br />
)<br />
The dual is infeasible.<br />
There is no primal improving direction.<br />
But:<br />
The dual is almost feasible.<br />
There is a primal almost improving direction.
IE496/4<br />
<strong>Imre</strong> Pólik<br />
Weak infeasibility<br />
Outline<br />
<strong>Feasibility</strong><br />
Review of LP<br />
Weak <strong>and</strong> strong<br />
infeasibility<br />
Farkas lemma for SDP<br />
Duality<br />
min C • X<br />
max b T y<br />
AX = b<br />
A ∗ y + S = C<br />
X ≽ 0 S ≽ 0<br />
The following are equivalent if the primal is infeasible:<br />
1 For every ε > 0 there is an X ≽ 0 such that<br />
‖AX − b‖ ≤ ε.<br />
2 For every ε > 0 there is a y ∈ R m <strong>and</strong> S ≽ 0 such that<br />
‖A ∗ y + S‖ ≤ ε <strong>and</strong> b T y = 1.<br />
The following are equivalent if the dual is infeasible:<br />
1 For every ε > 0 there is a y ∈ R m <strong>and</strong> S ≽ 0 such that<br />
‖A ∗ y + S − C‖ ≤ ε.<br />
2 For every ε > 0 there is an X ≽ 0 such that ‖AX‖ ≤ ε<br />
<strong>and</strong> C • X = −1.
IE496/4<br />
<strong>Imre</strong> Pólik<br />
Farkas lemma for SDP<br />
Outline<br />
<strong>Feasibility</strong><br />
Review of LP<br />
Weak <strong>and</strong> strong<br />
infeasibility<br />
Farkas lemma for SDP<br />
Duality<br />
min C • X<br />
max b T y<br />
AX = b<br />
A ∗ y + S = C<br />
X ≽ 0 S ≽ 0<br />
Theorem (Farkas lemma for SDP)<br />
The following two statements are NOT equivalent, but<br />
1 ⇒ 2:<br />
1 There is an X ∈ R n×n such that AX = b <strong>and</strong> X ≽ 0.<br />
2 There is no y ∈ R m such that A ∗ y ≼ 0 <strong>and</strong> b T y = 1.<br />
In other words, the existence of a primal (dual) improving<br />
direction implies dual (primal) infeasibility. The converse<br />
fails due to weak infeasibility.
IE496/4<br />
<strong>Imre</strong> Pólik<br />
LP <strong>duality</strong><br />
Outline<br />
<strong>Feasibility</strong><br />
Duality<br />
Review of LP <strong>duality</strong><br />
SDP <strong>duality</strong><br />
min c T x<br />
max b T y<br />
Ax = b<br />
A T y + s = c<br />
x ≥ 0 s ≥ 0<br />
Theorem (Duality of LP)<br />
If the primal (dual) problem is unbounded, then the<br />
dual (primal) is infeasible.<br />
If the primal (dual) problem is infeasible, then the dual<br />
(primal) problem is unbounded or infeasible.<br />
If both problems are feasible, then the optimal values<br />
are the same (there is no <strong>duality</strong> gap), <strong>and</strong> the optimum<br />
is attained for both problems.
IE496/4<br />
<strong>Imre</strong> Pólik<br />
Example 1, <strong>duality</strong> gap<br />
Outline<br />
<strong>Feasibility</strong><br />
Duality<br />
Review of LP <strong>duality</strong><br />
SDP <strong>duality</strong><br />
min<br />
⎛<br />
⎝ 0 0 0<br />
α 0 0<br />
0 0 0<br />
⎛<br />
⎝ 0 1 0<br />
0 0 0<br />
0 0 0<br />
⎛<br />
⎝ 0 0 1<br />
1 0 0<br />
0 1 0<br />
⎞<br />
⎠ • X<br />
⎞<br />
⎠ • X = 0<br />
⎞<br />
⎠ • X = 1<br />
X ≽ 0<br />
Dual optimum is 0, with y = 0<br />
Primal optimum is α, with<br />
x 11 = 1<br />
y 1<br />
⎛<br />
⎝<br />
0 0 0<br />
0 1 0<br />
0 0 0<br />
⎞ ⎛<br />
⎠ + y 2<br />
⎝<br />
1 0 0<br />
0 0 1<br />
0 1 0<br />
max y 2<br />
⎞<br />
⎛<br />
⎠ + S = ⎝<br />
S ≽ 0<br />
α 0 0<br />
0 0 0<br />
0 0 0<br />
What went wrong: dual feasible set is too small (Slater<br />
condition).<br />
⎞<br />
⎠
IE496/4<br />
<strong>Imre</strong> Pólik<br />
Example 2, unattained optimum<br />
Outline<br />
<strong>Feasibility</strong><br />
Duality<br />
Review of LP <strong>duality</strong><br />
SDP <strong>duality</strong><br />
(<br />
1 0<br />
min<br />
0 0<br />
(<br />
0 −1<br />
−1 0<br />
) ( )<br />
x11 x<br />
•<br />
12<br />
x 21 x 22<br />
)<br />
•<br />
( )<br />
x11 x 12<br />
= 2<br />
x 21 x 22<br />
(<br />
0 −1<br />
−1 0<br />
max 2y 1<br />
)<br />
(<br />
1 0<br />
y 1 + S =<br />
0 0<br />
X ≽ 0 S ≽ 0<br />
)<br />
Dual optimal value is 0, optimal dual solution is y 1 = 0.<br />
Primal optimal value is 0, but no optimal primal<br />
solution.<br />
What went wrong: dual feasible set is too small (Slater<br />
condition).
IE496/4<br />
<strong>Imre</strong> Pólik<br />
SDP <strong>duality</strong><br />
Outline<br />
<strong>Feasibility</strong><br />
Duality<br />
Review of LP <strong>duality</strong><br />
SDP <strong>duality</strong><br />
min C • X<br />
max b T y<br />
AX = b<br />
A ∗ y + S = C<br />
X ≽ 0 S ≽ 0<br />
Theorem (Duality for SDP)<br />
If the primal (dual) problem is strictly feasible, then the<br />
dual (primal) problem is solvable <strong>and</strong> the optimal values<br />
are equal.<br />
If both problems are strictly feasible, then they are both<br />
solvable <strong>and</strong> the optimal values are equal.
Change language