Convex Optimization — Assignment 7 - IFOR

Institute for Operations Research
ETH Zurich HG G21-23
Martin Ballerstein
Timm Oertel
Kevin Zemmer
martin.ballerstein@ifor.math.ethz.ch
timm.oertel@ifor.math.ethz.ch
kevin.zemmer@ifor.math.ethz.ch
Spring Term 2013
Convex Optimization — Assignment 7
Exercise 1: KKT Conditions with Equality Constraints
In this exercise we want to extend the KKT Theorem seen in the class to the more general setting
where the optimization problem also possesses linear equality constraints. More concretely, following
a similar idea of the proof seen in class, prove the following theorem:
Theorem (KKT Conditions for Convex Optimization III):
Consider the convex optimization problem
f ∗ := min{f(x) | g(x) b, l(x) = d},
where f is convex, g i are concave (i = 1, . . . , m), l j are linear (j = 1, . . . , M) and x ∈ R n .
Assume that Slater’s condition holds, i.e.:
Then x ∗ is an optimal solution of the problem iff
• (Feasibility) g(x ∗ ) b and l(x ∗ ) = d,
∃¯x s.t. g i (¯x) > b i , l j (¯x) = d j , ∀i, j.
• (KKT Condition) There exist h 0 ∈ ∂f(x ∗ ), h i ∈ ∂(−g i (x ∗ )), ξ j ∈ ∂l j (x ∗ ) as well as λ ∗ i ≥ 0, ν∗ j ∈ R
such that
m∑
M∑
h 0 + λ ∗ i h i + νj ∗ ξ j = 0,
• (Complementarity) λ ∗ i (b i − g i (x ∗ )) = 0, ∀i.
i=1
(Remark: The existence of a Slater point can be replaced by many other requirements, listed in the
literature as “Regularity Conditions” or “Constraint Qualifications”).
Verify the theorem by proving the following claims:
(a) x ∗ is an optimum of the problem iff x ∗ = arg min x Φ(x), where
j=1
Φ(x) := max{f(x) − f ∗ , b 1 − g 1 (x), . . . , b m − g m (x)} +
M∑
χ Lj (x),
L j := {x | l j (x) = d j }, and χ Lj is the characteristic function of the set L j for all j = 1, . . . , M.
(b) x ∗ = arg min x Φ(x) iff 0 ∈ ∂Φ(x ∗ ).
(c)
∑
0 ∈ ∂Φ(x ∗ ) iff there are h 0 ∈ ∂f(x ∗ ), h i ∈ ∂(−g i (x ∗ )) as well as β j ∈ R, α 0 , α i ≥ 0 with α 0 +
i∈I(x ∗ ) α i = 1 such that
0 = α 0 h 0 + ∑
M∑
α i h i + β j ξ j .
(d) Prove that α 0 ≠ 0.
i∈I(x ∗ )
(e) Reformulate 0 = α 0 h 0 + ∑ i∈I(x ∗ ) α ih i + ∑ M
j=1 β jξ j as 0 = h 0 + ∑ m
i=1 λ∗ i h i + ∑ M
j=1 ν∗ j ξ j and derive
the complementary constraint.
j=1
j=1

Exercise 2: Turning a quadratic problem into a linear one (part (f*) is a bonus of 5 points)
In a variety of different fields, we must solve an optimization problem of the following form:
W ∗ := min{〈F, x〉 : A(y)x = F, 〈1, y〉 = T, y ≥ 0}.
As an illustration, we can consider a standard problem in Truss Topology Design: We have a set of
n points, or nodes, on which we know that some forces will operate. Some of these points have to be
linked by rigid bars, so that the resulting mechanical structure can sustain all the given forces. The
problem consists in finding the size (that is, the thickness) of all these bars; of course, we can find that
some of these have a null thickness, meaning simply that we do not need them in our final structure.
We assume here that our structure is a two-dimensional one.
In the optimization problem above, the vector y ∈ IR N represent the (unknown) weight of each of
N potential bars, so that 〈1, y〉 = ∑ N
i=1 y i is the total weight of our construction, which we want to
be equal to T . The vector x ∈ IR 2n represents here the (small) displacement of each node due to
the forces. The vector F ∈ IR 2n is the given vector of (2D) external forces imposed at each node,
so that 〈F, x〉 is the total work done by the truss, which we want to minimize. Finally, the matrix
A(y) := ∑ N
i=1 y ia i a T i represents the compliance matrix of the truss described by the bars y; the vectors
a i are known. For simplicity, we assume that these a i ’s span IR 2n .
(a) Denoting ∆ T := {y ∈ IR N : 〈1, y〉 = T, y ≥ 0}, show that
W ∗ (i)
(ii)
= min max
y∈∆ T u∈IR
2n{2〈F,
u〉 − 〈u, A(y)u〉} = max
u∈IR 2n
(b) Show that max y∈∆T 〈u, A(y)u〉 = T max 1≤i≤N 〈a i , u〉 2 .
min {2〈F, u〉 − 〈u, A(y)u〉}.
y∈∆ T
(c) Show that S := {z ∈ IR 2n
| 1 = T max 1≤i≤N 〈a i , z〉 2 } is bounded and the boundary of a polyhedron
P with 0 ∈ int(P ).
(d) Show that W ∗ = max{2t〈F, z〉 − T t 2 max 1≤i≤N 〈a i , z〉 2 : t ∈ IR, 1 = T max 1≤i≤N 〈a i , z〉 2 }, which
equals
(
max{〈F, z〉 : √ ) 2
T max |〈a i, z〉| = 1} .
1≤i≤N
(e) Show that the latter problem can be solved via a linear optimization problem.
(f*) How can we reconstruct the solution (x ∗ , y ∗ ) of the original problem from this linear optimization
problem. (Strong hint: use the dual optimal solution too and look at all the relations that can
be written for it).
Please hand in your assignment no later than Thursday, April 18, 2013, 15:00 at HG G21 (“Convex Optimization” tray) or
during the class.
Certificate condition: 50% of the total marks of the exercises (each assignment will contain approx. 2 exercises).
2