Convex Optimization â Assignment 7 - IFOR
Convex Optimization â Assignment 7 - IFOR
Convex Optimization â Assignment 7 - IFOR
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
Institute for Operations Research<br />
ETH Zurich HG G21-23<br />
Martin Ballerstein<br />
Timm Oertel<br />
Kevin Zemmer<br />
martin.ballerstein@ifor.math.ethz.ch<br />
timm.oertel@ifor.math.ethz.ch<br />
kevin.zemmer@ifor.math.ethz.ch<br />
Spring Term 2013<br />
<strong>Convex</strong> <strong>Optimization</strong> — <strong>Assignment</strong> 7<br />
Exercise 1: KKT Conditions with Equality Constraints<br />
In this exercise we want to extend the KKT Theorem seen in the class to the more general setting<br />
where the optimization problem also possesses linear equality constraints. More concretely, following<br />
a similar idea of the proof seen in class, prove the following theorem:<br />
Theorem (KKT Conditions for <strong>Convex</strong> <strong>Optimization</strong> III):<br />
Consider the convex optimization problem<br />
f ∗ := min{f(x) | g(x) b, l(x) = d},<br />
where f is convex, g i are concave (i = 1, . . . , m), l j are linear (j = 1, . . . , M) and x ∈ R n .<br />
Assume that Slater’s condition holds, i.e.:<br />
Then x ∗ is an optimal solution of the problem iff<br />
• (Feasibility) g(x ∗ ) b and l(x ∗ ) = d,<br />
∃¯x s.t. g i (¯x) > b i , l j (¯x) = d j , ∀i, j.<br />
• (KKT Condition) There exist h 0 ∈ ∂f(x ∗ ), h i ∈ ∂(−g i (x ∗ )), ξ j ∈ ∂l j (x ∗ ) as well as λ ∗ i ≥ 0, ν∗ j ∈ R<br />
such that<br />
m∑<br />
M∑<br />
h 0 + λ ∗ i h i + νj ∗ ξ j = 0,<br />
• (Complementarity) λ ∗ i (b i − g i (x ∗ )) = 0, ∀i.<br />
i=1<br />
(Remark: The existence of a Slater point can be replaced by many other requirements, listed in the<br />
literature as “Regularity Conditions” or “Constraint Qualifications”).<br />
Verify the theorem by proving the following claims:<br />
(a) x ∗ is an optimum of the problem iff x ∗ = arg min x Φ(x), where<br />
j=1<br />
Φ(x) := max{f(x) − f ∗ , b 1 − g 1 (x), . . . , b m − g m (x)} +<br />
M∑<br />
χ Lj (x),<br />
L j := {x | l j (x) = d j }, and χ Lj is the characteristic function of the set L j for all j = 1, . . . , M.<br />
(b) x ∗ = arg min x Φ(x) iff 0 ∈ ∂Φ(x ∗ ).<br />
(c)<br />
∑<br />
0 ∈ ∂Φ(x ∗ ) iff there are h 0 ∈ ∂f(x ∗ ), h i ∈ ∂(−g i (x ∗ )) as well as β j ∈ R, α 0 , α i ≥ 0 with α 0 +<br />
i∈I(x ∗ ) α i = 1 such that<br />
0 = α 0 h 0 + ∑<br />
M∑<br />
α i h i + β j ξ j .<br />
(d) Prove that α 0 ≠ 0.<br />
i∈I(x ∗ )<br />
(e) Reformulate 0 = α 0 h 0 + ∑ i∈I(x ∗ ) α ih i + ∑ M<br />
j=1 β jξ j as 0 = h 0 + ∑ m<br />
i=1 λ∗ i h i + ∑ M<br />
j=1 ν∗ j ξ j and derive<br />
the complementary constraint.<br />
j=1<br />
j=1
Exercise 2: Turning a quadratic problem into a linear one (part (f*) is a bonus of 5 points)<br />
In a variety of different fields, we must solve an optimization problem of the following form:<br />
W ∗ := min{〈F, x〉 : A(y)x = F, 〈1, y〉 = T, y ≥ 0}.<br />
As an illustration, we can consider a standard problem in Truss Topology Design: We have a set of<br />
n points, or nodes, on which we know that some forces will operate. Some of these points have to be<br />
linked by rigid bars, so that the resulting mechanical structure can sustain all the given forces. The<br />
problem consists in finding the size (that is, the thickness) of all these bars; of course, we can find that<br />
some of these have a null thickness, meaning simply that we do not need them in our final structure.<br />
We assume here that our structure is a two-dimensional one.<br />
In the optimization problem above, the vector y ∈ IR N represent the (unknown) weight of each of<br />
N potential bars, so that 〈1, y〉 = ∑ N<br />
i=1 y i is the total weight of our construction, which we want to<br />
be equal to T . The vector x ∈ IR 2n represents here the (small) displacement of each node due to<br />
the forces. The vector F ∈ IR 2n is the given vector of (2D) external forces imposed at each node,<br />
so that 〈F, x〉 is the total work done by the truss, which we want to minimize. Finally, the matrix<br />
A(y) := ∑ N<br />
i=1 y ia i a T i represents the compliance matrix of the truss described by the bars y; the vectors<br />
a i are known. For simplicity, we assume that these a i ’s span IR 2n .<br />
(a) Denoting ∆ T := {y ∈ IR N : 〈1, y〉 = T, y ≥ 0}, show that<br />
W ∗ (i)<br />
(ii)<br />
= min max<br />
y∈∆ T u∈IR<br />
2n{2〈F,<br />
u〉 − 〈u, A(y)u〉} = max<br />
u∈IR 2n<br />
(b) Show that max y∈∆T 〈u, A(y)u〉 = T max 1≤i≤N 〈a i , u〉 2 .<br />
min {2〈F, u〉 − 〈u, A(y)u〉}.<br />
y∈∆ T<br />
(c) Show that S := {z ∈ IR 2n<br />
| 1 = T max 1≤i≤N 〈a i , z〉 2 } is bounded and the boundary of a polyhedron<br />
P with 0 ∈ int(P ).<br />
(d) Show that W ∗ = max{2t〈F, z〉 − T t 2 max 1≤i≤N 〈a i , z〉 2 : t ∈ IR, 1 = T max 1≤i≤N 〈a i , z〉 2 }, which<br />
equals<br />
(<br />
max{〈F, z〉 : √ ) 2<br />
T max |〈a i, z〉| = 1} .<br />
1≤i≤N<br />
(e) Show that the latter problem can be solved via a linear optimization problem.<br />
(f*) How can we reconstruct the solution (x ∗ , y ∗ ) of the original problem from this linear optimization<br />
problem. (Strong hint: use the dual optimal solution too and look at all the relations that can<br />
be written for it).<br />
Please hand in your assignment no later than Thursday, April 18, 2013, 15:00 at HG G21 (“<strong>Convex</strong> <strong>Optimization</strong>” tray) or<br />
during the class.<br />
Certificate condition: 50% of the total marks of the exercises (each assignment will contain approx. 2 exercises).<br />
2