Algorithm Design

More documents

Recommendations

Info

666 Chapter 12 Local Search to every node of the form yj. Then there are two local minima, corresponding to the vertex covers [xl ..... x~} and [Yl ..... Ye}. Which one is discovered by a run of gradient descent is ent~ely determined by whether it first deletes an element of the form xi or yj. With more complicated graphs, it’s often a useful exercise to think about the kind of landscape they induce; and conversely, one sometimes may look at a landscape and consider whether there’s a graph that gives rise to something like it. For example, what kind of graph might yield a Vertex Cover instance with a landscape like the iagged furmel in Figure 12.37 One such graph is simply an n-node path, where’n is an odd number, with nodes labeled vl, vz ..... Vn in order. The unique minimum vertex cover S* consists of all nodes vi where i is even. But there are many local optima. For example, consider the vertex cover {v2, v3, vs, v6, v8, v9 .... } in which every third node is omitted. This is a vertex cover that is significantly larger than S*; but there’s no way to delete any node from it while stil! covering all edges. Indeed, it’s very hard for gradient descent to find the minimum vertex cover S* starting from the kill vertex set ~: Once it’s deleted just a single node vi with an even value of i, it’s lost the chance to find the global optimum S*. Thus the even/odd parity distinction in the nodes captures a plethora of different wrong turns in the local search, and hence gives the overall funnel its jagged character. Of course, there is not a direct correspondence between the ridges in the drawing and the local optima; as we warned above, Figure 12.3 is ultimately just a cartoon rendition of what’s going on. But we see that even for graphs that are structurally very simple, gradient descent is much too straightforward a local search algorithm. We now look at some more refined local search algorithms that use the same type of neighbor relation, but include a method for "escaping" from local minima. 12.2 The Metropolis Algorithm and Simulated Annealing The first idea for an improved local search algorithm comes from the work of Metropolis, Rosenbluth, Rosenbluth, Teller, and Teller (1953). They considered the problem of simulating the behavior of a physical system according to principles of statistical mechanics. A basic model from this field asserts that the probability of finding a physical system in a state with energy E is proportional to the Gibbs-Boltzmann function e -EI(kT) , where T > 0 is the temperature and k > 0 is a constant. Let’s look at this flmction. For any temperature T, the flmction is monotone decreasing in the energy E, so this states that a physical 12.2 The Metropolis Algorithm and Simulated Annealing system is more likely to be in a lower energy state than in a high energy state. Now let’s consider the effect of the temperature T. When T is small, the probability for a low-energy state is significantly larger than the probability for a high-energy state. However, if the temperature is large, then the difference between these two probabilities is very sma!l, and the system is almost equally likely to be in any state. The Metropoli~s Algorithm Metropolis et al. proposed the following method for performing step-by-step simulation of a system at a fixed temperature T. At all times, the simulation maintains a current state of the system and tries to produce a new state by applying a perturbation to this state. We’ll assume that we’re only interested in states of the system that are "reachable" from some fixed initial state by a sequence of small perturbations, and we’ll assume that there is_ only a finite set ~ of such states. In a single step, we first generate a small random perturbation to the current state S of the system, resulting in a new state S’. LetE(S) and E(S’) denote the energies of S and S’, respectively. If E(S’) < E(S), then we update the current state to be S’. Otherwise let AE = E(S’) E(S) > 0. We update the current state to be S’ with probability e -AE/(kT) , and otherwise leave the current state at S. Metropolis et al. proved that their simulation algorithm has the following property. To prevent too long a digression, we omit the proof; it is actually a direct consequence of some basic facts about random walks. (12.2) Let Z ~- E e-E(S)/(kT)" For a state S, let is(t) denote the fraction of the first t steps in which the state of the simulation is in S. Then the limit of is(t) as t approaches oo is, with probability approaching 1, equal to ½ ¯ e -E(s)/(kT) . This is exactly the sort of fact one wants, since it says that the simulation spends roughly the correct amount of time in each state, according to the Gibbs-Boltzmann equation. If we want to use this overall scheme to design a local search algorithm for minimization problems, we can use the analogies of Section 12.1 in which states of the system are candidate solutions, with energy corresponding to cost. We then see that the operation of the Metropolis Algorithm has a very desirable pair of features in a local search algorithm: It is biased toward "downhill" 667
668 Chapter 12 Local Search moves but will also accept "uphill" moves with smaller probability. In this way, it is able to make progress even when situated in a local minimum. Moreover, as expressed in (12.2), it is globa!ly biased toward lower-cost solutions. Here is a concrete formulation of the Metropolis Algorithm for a minimization problem. Staxt with an initial solution So, and constants k and T In one step: Let S be the current solution Let S’ be chosen uniformly at random from the neighbors of S If c(S’)
Page 2 and 3:
Cornell University Boston San Franc
Page 4 and 5:
Contents About the Authors Preface
Page 6 and 7:
X Contents 9 10 Extending the Limit
Page 8 and 9:
xiv Preface problems out in the wor
Page 10 and 11:
Preface Chapters 2 and 3 also prese
Page 12 and 13:
xxii Preface Preface xxiii Sue Whit
Page 14 and 15:
2 Chapter 1 Introduction: Some Repr
Page 16 and 17:
6 Chapter 1 Introduction: Some Repr
Page 18 and 19:
10 Chapter 1 Introduction: Some Rep
Page 20 and 21:
14 Chapter ! Introduction: Some Rep
Page 22 and 23:
Page 24 and 25:
Page 26 and 27:
Page 28 and 29:
30 Chapter 2 Basics of Algorithm An
Page 30 and 31:
34 n= I0 n=30 n=50 = 100 = 1,000 10
Page 32 and 33:
Page 34 and 35:
Page 36 and 37:
Page 38 and 39:
Page 40 and 41:
Page 42 and 43:
Page 44 and 45:
Page 46 and 47:
Page 48 and 49:
Page 50 and 51:
74 Chapter 3 Graphs 3.1 Basic Defin
Page 52 and 53:
78 Chapter 3 Graphs Rooted trees ar
Page 54 and 55:
82 Chapter 3 Graphs Current compone
Page 56 and 57:
86 Chapter 3 Graphs Proof. Suppose
Page 58 and 59:
90 Chapter 3 Graphs Two of the simp
Page 60 and 61:
94 Chapter 3 Graphs most ~u nv = 2m
Page 62 and 63:
98 Chapter 3 Graphs It is important
Page 64 and 65:
102 Chapter 3 Graphs ordering, that
Page 66 and 67:
106 Chapter 3 Graphs We can already
Page 68 and 69:
110 Chapter 3 Graphs Exercises 111
Page 70 and 71:
Greedy A~gorithms In Wall Street, t
Page 72 and 73:
118 Chapter 4 Greedy Mgofithms o In
Page 74 and 75:
122 Chapter 4 Greedy Algorithms Our
Page 76 and 77:
126 Chapter 4 Greedy Algorithms Len
Page 78 and 79:
Before swapping: After swapping: d
Page 80 and 81:
134 Chapter 4 Greedy Algorithms wit
Page 82 and 83:
138 Chapter 4 Greedy Algorithms 4.4
Page 84 and 85:
142 Chapter 4 Greedy Algorithms 4.5
Page 86 and 87:
146 Chapter 4 Greedy Algorithms (e
Page 88 and 89:
150 Chapter 4 Greedy Algorithms her
Page 90 and 91:
154 Chapter 4 Greedy Algorithms we
Page 92 and 93:
158 Chapter 4 Greedy Algorithms spa
Page 94 and 95:
162 Chapter 4 Greedy Algorithms ~ T
Page 96 and 97:
166 Chapter 4 Greedy Algorithms g2(
Page 98 and 99:
170 Chapter 4 Greedy Algorithms Sha
Page 100 and 101:
174 Chapter 4 Greedy Algorithms ...
Page 102 and 103:
178 Chapter 4 Greedy Algorithms Fig
Page 104 and 105:
182 Chapter 4 Greedy Algorithms The
Page 106:
186 Chapter 4 Greedy Algorithms Sol
Page 109 and 110:
192 Chapter 4 Greedy Algorithms 8.
Page 111 and 112:
Page 113 and 114:
Page 115 and 116:
204 Chapter 4 Greedy Algorithms Not
Page 117 and 118:
Chapter Divide artd Cortquer Divide
Page 119 and 120:
212 Chapter 5 Divide and Conquer Th
Page 121 and 122:
216 cn time plus recursive calls Ch
Page 123 and 124:
220 Chapter 5 Divide and Conquer mo
Page 125 and 126:
224 Chapter 5 Divide and Conquer El
Page 127 and 128:
228 Chapter 5 Divide and Conquer 5.
Page 129 and 130:
232 Chapter 5 Divide and Conquer (a
Page 131 and 132:
we’ll just give a simple example
Page 133 and 134:
240 Chapter 5 Divide and Conquer po
Page 135 and 136:
244 Chapter 5 Divide and Conquer Th
Page 137 and 138:
248 Chapter 5 Divide and Conquer It
Page 139 and 140:
252 Chapter 6 Dynamic Programming b
Page 141 and 142:
256 Chapter 6 Dynamic Programming 6
Page 143 and 144:
260 Index 1 2 3 4 5 6 L~ I = 2 Chap
Page 145 and 146:
264 Chapter 6 Dynamic Programming w
Page 147 and 148:
268 Chapter 6 Dynamic Programming t
Page 149 and 150:
272 Chapter 6 Dynamic Programming s
Page 151 and 152:
Page 153 and 154:
280 Chapter 6 Dynamic Programming t
Page 155 and 156:
284 Figure 6.18 The OPT values for
Page 157 and 158:
288 Chapter 6 Dynamic Programming U
Page 159 and 160:
292 (a) Figure 6.21 (a) With negati
Page 161 and 162:
296 Chapter 6 Dynamic Programming i
Page 163 and 164:
300 Chapter 6 Dynamic Programming p
Page 165 and 166:
304 Chapter 6 Dynamic Programming e
Page 167 and 168:
308 Chapter 6 Dynamic Programming o
Page 169 and 170:
312 Chapter 6 Dynamic Programming E
Page 171 and 172:
316 Chapter 6 Dynamic Programming T
Page 173 and 174:
320 Chapter 6 Dynamic Programming (
Page 175 and 176:
Page 177 and 178:
328 Chapter 6 Dynamic Programming T
Page 179 and 180:
Page 181 and 182:
336 Chapter 6 Dynamic Programming h
Page 183 and 184:
54O Chapter 7 Network Flow define f
Page 185 and 186:
344 Chapter 7 Network Flow This aug
Page 187 and 188:
348 Chapter 7 Network Flow Proof. ~
Page 189 and 190:
352 Chapter 7 Network Flow In the n
Page 191 and 192:
356 Chapter 7 Network Flow (7.19) T
Page 193 and 194:
360 Chapter 7 Network Flow preflow
Page 195 and 196:
364 Chapter 7 Network Flow Heights
Page 197 and 198:
368 Chapter 7 Network Flow ~ The Pr
Page 199 and 200:
372 Chapter 7 Network Flow that are
Page 201 and 202:
376 Chapter 7 Network Flow I~low ar
Page 203 and 204:
380 Chapter 7 Network Flow Figure 7
Page 205 and 206:
384 Chapter 7 Network Flow Lower bo
Page 207 and 208:
388 BOS 6 DCA 7 DCA 8 Chapter 7 Net
Page 209 and 210:
392 Chapter 7 Network Flow natural
Page 211 and 212:
396 Chapter 7 Network Flow 7.11 Pro
Page 213 and 214:
400 Chapter 7 Network Flow 7.12 Bas
Page 215 and 216:
404 Chapter 7 Network Flow to cross
Page 217 and 218:
4O8 Chapter 7 Network Flow Why are
Page 219 and 220:
412 Chapter 7 Network Flow ProoL Th
Page 221 and 222:
416 Chapter 7 Network Flow Figure 7
Page 223 and 224:
420 Chapter 7 Network Flow 11. Now
Page 225 and 226:
424 Figure 7.29 An instance of Cove
Page 227 and 228:
428 Chapter 7 Network Flow 22. This
Page 229 and 230:
432 Chapter 7 Network Flow generall
Page 231 and 232:
436 Chapter 7 Network Flow (a) (b)
Page 233 and 234:
440 Chapter 7 Network Flow going to
Page 235 and 236:
Chapter 7 Network Flow 44. 45. Give
Page 237 and 238:
448 Chapter 7 Network Flow 51. The
Page 239 and 240:
452 Chapter 8 NP and Computational
Page 241 and 242:
Page 243 and 244:
Page 245 and 246:
Page 247 and 248:
Page 249 and 250:
472 Chapter 8 NP and Computation!!
Page 251 and 252:
Page 253 and 254:
Page 255 and 256:
Page 257 and 258:
Page 259 and 260:
Page 261 and 262:
Page 263 and 264:
5OO Chapter 8 NP and Computational
Page 265 and 266:
Page 267 and 268:
Page 269 and 270:
Page 271 and 272:
Page 273 and 274:
Page 275 and 276:
Page 277 and 278:
Page 279 and 280:
532 Chapter 9 PSPACE: A Class of Pr
Page 281 and 282:
Page 283 and 284:
Page 285 and 286:
Page 287 and 288:
Page 289 and 290:
Page 291 and 292:
556 Chapter 10 Extending the Limits
Page 293 and 294:
560 Chapter 10 Extending the Limits
Page 295 and 296: 564 Chapter 10 Extending the Limits
Page 303 and 304: 58O Chapter 10 Extending the Limits
Page 311 and 312: 596 Figure 10.12 A triangulated cyc
Page 313 and 314: 600 Chapter 11 Approximation Algori
Page 327 and 328: 628 Chapter 11 Approximation Mgorit
Page 331 and 332: 636 Chapter !! Approximation Algori
Page 333 and 334: 640 Chapter 11 Approximation Mgofit
Page 341 and 342: 656 Chapter 11 Approximation Mgorit
Page 345: 664 Chapter 12 Local Search basic p
Page 349 and 350: 672 Chapter 12 Local Search of all
Page 351 and 352: 676 Chapter 12 Local Search 12.4 Ma
Page 353 and 354: 680 Chapter 12 Local Search saw ear
Page 355 and 356: 684 Chapter 12 Local Search utting
Page 357 and 358: 688 Chapter 12 Local Search this di
Page 359 and 360: 692 Chapter 12 Local Search sides a
Page 361 and 362: 696 Chapter 12 Local Search Of cour
Page 363 and 364: 7OO Chapter 12 Loca! Search and for
Page 365 and 366: 7O4 Chapter 12 Local Search A previ
Page 367 and 368: 708 Chapter 13 Randomized Algorithm
Page 375 and 376: 724 Chapter 13 Randomized Mgorithms
Page 381 and 382: 736 Chapter 13 Randomized Mgofithms
Page 397 and 398:
768 Chapter 13 Randomized Algorithm
Page 399 and 400:
Page 401 and 402:
Page 403 and 404:
Page 405 and 406:
Page 407 and 408:
Page 409 and 410:
792 Chapter !3 Randomized Algorithm
Page 411 and 412:
796 Epilogue: Algorithms That Run F
Page 413 and 414:
800 Epilogue: Algorithms That Run F
Page 415 and 416:
8O4 Epilogue: Algorithms That Run F
Page 417 and 418:
808 References L. R. Ford. Network
Page 419 and 420:
812 References M. Mitzenmacher and
Page 421 and 422:
816 Index Approximation algorithms
Page 423 and 424:
820 Cushions in packet switching, 8
Page 425 and 426:
824 Index Graphs (cont.) queues and
Page 427 and 428:
828 Index Mapping genomes, 279, 521
Page 429 and 430:
832 Index Index 833 Preflow-Push Ma
Page 431 and 432:
836 Index Stale items in randomized
show all

Algorithm Design

Create successful ePaper yourself

Delete template?

Save as template?