Exercises
Exercises
Exercises
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
Discrete (and Continuous) Optimization<br />
WI4 131<br />
Kees Roos<br />
Technische Universiteit Delft<br />
Faculteit Electrotechniek, Wiskunde en Informatica<br />
Afdeling Informatie, Systemen en Algoritmiek<br />
e-mail: C.Roos@ewi.tudelft.nl<br />
URL: http://www.isa.ewi.tudelft.nl/˜roos<br />
November – December, A.D. 2004
Course Schedule<br />
1. Formulations (18 pages)<br />
2. Optimality, Relaxation, and Bounds (10 pages)<br />
3. Well-solved Problems (13 pages)<br />
4. Matching and Assigments (10 pages)<br />
5. Dynamic Programming (11 pages)<br />
6. Complexity and Problem Reduction (8 pages)<br />
7. Branch and Bound (17 pages)<br />
8. Cutting Plane Algorithms (21 pages)<br />
9. Strong Valid Inequalities (22 pages)<br />
10. Lagrangian Duality (14 pages)<br />
11. Column Generation Algorithms (16 pages)<br />
12. Heuristic Algorithms (15 pages)<br />
13. From Theory to Solutions (20 pages)<br />
Optimization Group 1
Capter 12<br />
Heuristic Algorithms<br />
Optimization Group 2
Exercise 12.7.1<br />
Apply the different heuristics to an instance of STSP with the following distance matrix<br />
⎛<br />
⎜<br />
⎝<br />
- 28 57 72 81 85 80<br />
28 - 28 45 54 57 63<br />
57 28 - 20 3 28 57<br />
72 45 20 - 10 20 72<br />
81 54 3 10 - 22 81<br />
85 57 28 20 22 - 63<br />
80 63 57 72 81 63 -<br />
Pure Greedy Solution: Order the edges according to nondecreasing cost, and seek to use<br />
them in this order.<br />
3<br />
2<br />
1<br />
4 5<br />
7<br />
6<br />
Heuristic tour<br />
#1<br />
step arc length<br />
1 (3,5) 3 accept<br />
2 (4,5) 10 accept<br />
3 (3,4) 20 subtour<br />
4 (4,6) 20 accept<br />
5 (5,6) 22 subtour<br />
6 (1,2) 28 accept<br />
7 (2,3) 28 accept<br />
8 (1,7) 80 forced<br />
9 (6,7) 63 forced<br />
⎞<br />
⎟<br />
⎠<br />
The length of the created tour is z 1 =<br />
3+10+20+28+28+80+63 =<br />
232. The same tour is obtained by using<br />
the insertion heuristic using nearest<br />
neighbor (starting at any node) and farthest<br />
neighbor (starting at node 3, else<br />
the length becomes 251). The tour is<br />
2-optimal.<br />
Optimization Group 3
Exercise 12.7.1 (other heuristic solution)<br />
⎛<br />
⎜<br />
⎝<br />
- 28 57 72 81 85 80<br />
28 - 28 45 54 57 63<br />
57 28 - 20 3 28 57<br />
72 45 20 - 10 20 72<br />
81 54 3 10 - 22 81<br />
85 57 28 20 22 - 63<br />
80 63 57 72 81 63 -<br />
Let us solve the assignment relaxation. We use the same technique as in Exercise 4.5.9. We<br />
first construct a dual feasible solution (since we minimize we invert the weights).<br />
⎞<br />
⎟<br />
⎠<br />
-28 -28 -3 -10 -3 -20 -57<br />
0 - -28 -57 -72 -81 -85 -80<br />
0 -28 - -28 -45 -54 -57 -63<br />
0 -57 -28 - -20 -3 -28 -57<br />
0 -72 -45 -20 - -10 -20 -72<br />
0 -81 -54 -3 -10 - -22 -81<br />
0 -85 -57 -28 -20 -22 - -63<br />
0 -80 -63 -57 -72 -81 -63 -<br />
⇒<br />
-190 -28 -28 -3 -10 -3 -20 -57<br />
0 - 0 -54 -62 -78 -65 -23<br />
0 0 - -25 -35 -51 -37 -6<br />
0 -29 0 - -10 0 -8 0<br />
0 -44 -17 -17 - -7 0 -15<br />
0 -53 -26 0 0 - -2 -24<br />
-6 -51 -23 -19 -4 -13 - 0<br />
-35 -17 0 -19 -27 -43 -8 -<br />
Thus u = (0, 0, 0, 0, 0, −6, −28), v = (−28, −28, −3, −10, −3, −20, −57) is<br />
the initial dual feasible solution and w = −183.<br />
Optimization Group 4
Exercise 12.7.1 (cont.)<br />
-190 -28 -28 -3 -10 -3 -20 -57<br />
0 - 0 -54 -62 -78 -65 -23<br />
0 0 - -25 -35 -51 -37 -6<br />
0 -29 0 - -10 0 -8 0<br />
0 -44 -17 -17 - -7 0 -15<br />
0 -53 -26 0 0 - -2 -24<br />
-6 -51 -23 -19 -4 -13 - 0<br />
-35 -17 0 -19 -27 -43 -8 -<br />
-198 -28 -20 -3 -10 -3 -20 -57<br />
-8 - 0 -46 -54 -70 -57 -15<br />
0 0 - -25 -35 -51 -37 -6<br />
0 -29 -8 - -10 0 -8 0<br />
0 -44 -25 -17 - -7 0 -15<br />
0 -53 -34 0 0 - -2 -24<br />
-6 -51 -31 -19 -4 -13 - 0<br />
-43 -9 0 -11 -19 -35 0 -<br />
-205 -28 -13 -3 -10 -3 -13 -57<br />
-15 - 0 -39 -47 -63 -57 -8<br />
0 0 - -25 -35 -51 -44 -6<br />
0 -29 -15 - -10 0 -15 0<br />
-7 -37 -25 -10 - 0 0 -8<br />
0 -53 -41 0 0 - -9 -24<br />
-6 -51 -38 -19 -4 -13 - 0<br />
-50 -2 0 -4 -12 -28 0 -<br />
*<br />
7<br />
6<br />
7’<br />
6’<br />
⇒<br />
*<br />
7<br />
6<br />
7’<br />
6’ *<br />
⇒<br />
*<br />
*<br />
7<br />
6<br />
7’ *<br />
6’ *<br />
5<br />
5’<br />
5<br />
5’<br />
5<br />
5’ *<br />
4<br />
4’<br />
*<br />
4<br />
4’<br />
*<br />
4<br />
4’<br />
3<br />
3’<br />
3<br />
3’<br />
*<br />
3<br />
3’<br />
2<br />
2’ *<br />
2<br />
2’ *<br />
2<br />
2’ *<br />
*<br />
1<br />
1’<br />
*<br />
1<br />
1’<br />
*<br />
1<br />
1’<br />
δ = 8<br />
δ = 7<br />
δ = 2<br />
Optimization Group 5
Exercise 12.7.1 (cont.)<br />
-205 -28 -13 -3 -10 -3 -13 -57<br />
-15 - 0 -39 -47 -63 -57 -8<br />
0 0 - -25 -35 -51 -44 -6<br />
0 -29 -15 - -10 0 -15 0<br />
-7 -37 -25 -10 - 0 0 -8<br />
0 -53 -41 0 0 - -9 -24<br />
-6 -51 -38 -19 -4 -13 - 0<br />
-50 -2 0 -4 -12 -28 0 -<br />
-207 -28 -11 -3 -10 -1 -11 -55<br />
-17 - 0 -37 -45 -63 -57 -8<br />
0 0 - -25 -35 -53 -46 -8<br />
-2 -27 -15 - -8 0 -15 0<br />
-9 -35 -25 -8 - 0 0 -8<br />
0 -53 -43 0 0 - -11 -26<br />
-8 -49 -38 -17 -2 -13 - 0<br />
-52 0 0 -2 -10 -28 0 -<br />
-209 -28 -11 -1 -8 -1 -11 -55<br />
-19 - 0 -35 -43 -63 -57 -8<br />
-2 0 - -23 -33 -53 -46 -8<br />
-4 -27 -15 - -6 0 -15 0<br />
-11 -35 -25 -6 - 0 0 -8<br />
0 -55 -45 0 0 - -13 -28<br />
-10 -49 -38 -15 0 -13 - 0<br />
-54 0 0 0 -8 -28 0 -<br />
*<br />
*<br />
7<br />
6<br />
7’ *<br />
6’ *<br />
⇒<br />
*<br />
*<br />
7<br />
6<br />
7’ *<br />
6’ *<br />
⇒<br />
*<br />
*<br />
7<br />
6<br />
7’ *<br />
6’ *<br />
5<br />
5’ *<br />
5<br />
5’ *<br />
5<br />
5’ *<br />
*<br />
4<br />
4’<br />
*<br />
4<br />
4’<br />
*<br />
4<br />
4’<br />
*<br />
3<br />
3’<br />
*<br />
3<br />
3’<br />
*<br />
3<br />
3’ *<br />
2<br />
2’ *<br />
*<br />
2<br />
2’ *<br />
*<br />
2<br />
2’ *<br />
*<br />
1<br />
1’<br />
*<br />
1<br />
1’ *<br />
*<br />
1<br />
1’ *<br />
δ = 2<br />
δ = 2<br />
Optimization Group 6
Exercise 12.7.1 (cont.)<br />
-209 -28 -11 -1 -8 -1 -11 -55<br />
-19 - 0 -35 -43 -63 -57 -8<br />
-2 0 - -23 -33 -53 -46 -8<br />
-4 -27 -15 - -6 0 -15 0<br />
-11 -35 -25 -6 - 0 0 -8<br />
0 -55 -45 0 0 - -13 -28<br />
-10 -49 -38 -15 0 -13 - 0<br />
-54 0 0 0 -8 -28 0 -<br />
7<br />
6<br />
5<br />
4<br />
3<br />
2<br />
1<br />
7’<br />
6’<br />
5’<br />
4’<br />
3’<br />
2’<br />
1’<br />
Optimal matching. Length<br />
209.<br />
The optimal matching yields two sub tours: 1 → 2 → 1<br />
and 3 → 5 → 4 → 6 → 7 → 3, with lengths<br />
28+28 = 56 and 3+10+20+63+57 = 153,<br />
respectively. This is a lower bound for an optimal tour.<br />
A heuristic approach might be the following: open the<br />
subtours by removing one edge from each of them and<br />
combine the two pieces to a tour by adding two edges (in<br />
one of the two possible ways). E.g., removing one edge<br />
{1,2} from the first subroute and edge {3,7} form the<br />
second, and adding the edges {1,7} and {2,3} we get<br />
the tour 3 → 5 → 4 → 6 → 7 → 1 → 2 → 3 which<br />
has length 209 − 28 − 57 + 80 + 28 = 232. This<br />
is exactly the same 2-optimal tour that we have found<br />
earlier by using the insertion heuristic.<br />
Optimization Group 7
Exercise 12.7.2<br />
Apply greedy and local neighborhood heuristics to an instance of the problem of most profitably<br />
allocating clients to at most K depots with m = 7 clients, n = 6 potential depots, K = 3,<br />
and the following cost matrix:<br />
(c ij ) =<br />
⎛<br />
⎜<br />
⎝<br />
2 3 7 3 6 1<br />
3 1 1 8 10 4<br />
6 2 3 1 2 7<br />
8 1 4 6 2 3<br />
4 4 3 3 4 3<br />
2 8 3 6 3 2<br />
6 5 3 2 7 4<br />
The column sums in the cost matrix are respectively 31,24,24,29,34 and 24. So from<br />
a greedy point of view the depots 2, 3 and 6 are most profitable. Let us start our greedy<br />
solution with S = {2}. We choose the second depot from the other depots so as to minimize<br />
the cost. When adding depot 1 the cost becomes 2+1+2+1+4+2+5 = 17. When<br />
adding the other (single) depots these costs become 16 for depot 3, 17 for depot 4, 19 for<br />
depot 5 and 14 for depot 6. So our new solution is S = {2,6}, with cost 14. Adding a<br />
third depot to this set the costs become 14 for depot 1, 13 for depot 3, 11 for depot 4 and<br />
14 for depot 5. Thus we choose to add depot 4, which gives S = {2,4,6}, with cost 11.<br />
⎞<br />
⎟<br />
⎠<br />
Optimization Group 8
Exercise 12.7.2 (cont.)<br />
(c ij ) =<br />
⎛<br />
⎜<br />
⎝<br />
2 3 7 3 6 1<br />
3 1 1 8 10 4<br />
6 2 3 1 2 7<br />
8 1 4 6 2 3<br />
4 4 3 3 4 3<br />
2 8 3 6 3 2<br />
6 5 3 2 7 4<br />
We found the greedy solution S = {2,4,6}, with cost 11. Let us apply a local search to<br />
this solution. Since the number of depots is fixed (K = 3), a natural neighborhood consists<br />
of all sets T arising by replacing one depot in S by a depot not belonging to S:<br />
T cost(T)<br />
{1,4,6} 15<br />
{3,4,6} 13<br />
{4,5,6} 15<br />
{1,2,6} 14<br />
{2,3,6} 13<br />
{2,5,6} 14<br />
{1,2,4} 12<br />
{2,3,4} 14<br />
{2,4,5} 14<br />
It is clear from this table that S is locally optimal.<br />
⎞<br />
⎟<br />
⎠<br />
Optimization Group 9
Exercise 12.7.3<br />
Devise a greedy heuristic for the set covering problem.<br />
Solution: The set covering problem is given by<br />
min<br />
⎧<br />
⎨<br />
∑<br />
⎩<br />
j∈N<br />
c j x j :<br />
∑<br />
j∈N<br />
a ij x j ≥ 1 for i ∈ M, x ∈ {0,1} n ⎫<br />
⎬<br />
⎭ ,<br />
where N = {1, . . . , n}, M = {1, . . . , m}, and a ij ∈ {0,1} for i ∈ M and j ∈ N.<br />
Let s j be the 0-1 vector consisting of all a ij ’s, i ∈ M and S j the subset of M which has<br />
s j as incidence vector. If x j = 1, then all constraints ∑ j∈N a ij x j ≥ 1 with i ∈ S j are<br />
satisfied. In other words, if x j = 1 then all elements of the subset S j of M are covered.<br />
The contribution to the cost function is c j . The cost per covered element is c j<br />
. So we start<br />
by putting x j = 1 for the (or an) index j that minimizes<br />
c j<br />
|S j |<br />
|S j |<br />
. Without loss of generality<br />
we assume that j = 1. Since the elements in S 1 are now covered, we replace each S j with<br />
j ≥ 2 by Sj 1 = S c<br />
j \ S 1 , and put x j = 1 for the (or an) index j ≥ 2 that minimizes j<br />
Without loss of generality we assume that j = 2. Then we replace each S j with j ≥ 3 by<br />
Sj 2 = S1 j \ S c<br />
2, and put x j = 1 for the (or an) index j ≥ 2 that minimizes j ∣, and so on.<br />
We proceed until all elements of M are covered, and put x j = 0 for the remaining variables.<br />
∣<br />
∣S 2 j<br />
∣<br />
∣<br />
∣S 1 j<br />
Optimization Group 10<br />
.<br />
∣
Exercise 12.7.4<br />
Consider the problem of finding a maximum cardinality matching in a graph G = (V, E). A<br />
mtching M ⊆ E is called maximal if M ∪ {e} is not a matching for any e ∈ E \ M. Let<br />
z be the size of a maximum matching and z H the size of a maximal matching. Show that<br />
z H ≥ 1 2 z.<br />
1<br />
2<br />
3<br />
4<br />
5<br />
6<br />
1’<br />
2’<br />
3’<br />
4’<br />
5’<br />
6’<br />
Example of a<br />
maximal matching.<br />
Solution: Let M be a maximum cardinality matching in G.<br />
Then |M| = z. We know that the size of a maximum cardinality<br />
matching is equal to the minimal size of a covering of the edges<br />
by nodes. So any covering of the edges by nodes contains at least<br />
z nodes.<br />
Now consider the end points of the edges in a maximal matching<br />
M ′ . These points necessarily cover all the edges in G; because<br />
any not covered edge e could be used to extend M ′ to a matching<br />
M ′ ∪ {e}, which is impossible since M ′ is maximal. Hence, the<br />
end points form a covering of the edges by nodes. The number of<br />
the end points being 2 ∣∣ M ′∣ ∣, we have 2 ∣∣ M ′∣∣ ≥ z, which implies<br />
the desired inequality.<br />
Optimization Group 11
Consider the 0-1 knapsack problem:<br />
⎧<br />
⎨<br />
z = max<br />
⎩<br />
Exercise 12.7.5<br />
n∑<br />
c j x j :<br />
j=1<br />
⎫<br />
n∑<br />
⎬<br />
a j x j ≤ b, x ∈ {0,1} n ⎭<br />
j=1<br />
with 0 < a j ≤ b for j ∈ N = {1, . . . , n}. Consider a greedy heuristic that chooses the better of the integer<br />
round down of the linear relaxation solution and the best solution in which just one variable is set to 1. Show<br />
that z G ≥ 1 2 z.<br />
Solution: Without loss of generality we assume that the variables are ordered such that c j<br />
a j<br />
is non-increasing.<br />
Let r be the largest index such that ∑ r<br />
j=1 a j ≤ b. Then the solution of the linear relaxation is given by<br />
x 1 = . . . = x r = 1, x r+1 = b − ∑ r<br />
j=1 a j<br />
, x r+2 = . . . = x n = 0<br />
a r+1<br />
whence integer round down of the linear relaxation solution yields the solution x 1 with<br />
r∑<br />
x 1 1 = . . . = x 1 r = 1, x 1 r+1 = . . . = x1 n = 0, z 1 = c T x 1 = c j .<br />
The best solution x 2 in which just one variable is set to 1 chooses the index k such that c k ≥ c j for all j and<br />
takes x 2 k = 1 and x2 j = 0 for j ≠ k, with value z2 = c k . Note that the value of the linear relaxation solution<br />
provides an upper bouns for z. Hence, using b − ∑ r<br />
j=1 a j < a r+1 , we may write<br />
z ≤ ¯z =<br />
r∑<br />
j=1<br />
c j + c r+1<br />
b − ∑ r<br />
j=1 a j<br />
a r+1<br />
proving the desired inequality.<br />
≤<br />
r∑<br />
c j + c r+1 ≤<br />
j=1<br />
j=1<br />
⎛<br />
r∑<br />
c j + c k ≤ 2max ⎝<br />
j=1<br />
r∑<br />
j=1<br />
c j , c k<br />
⎞<br />
⎠ = 2z G ,<br />
Optimization Group 12<br />
•