Data Structures and Algorithm Analysis - Computer Science at ...

More documents

Recommendations

Info

15Lower BoundsHow do I know if I have a good algorithm to solve a problem? If my algorithm runsin Θ(n log n) time, is that good? It would be if I were sorting the records storedin an array. But it would be terrible if I were searching the array for the largestelement. The value of an algorithm must be determined in relation to the inherentcomplexity of the problem at hand.In Section 3.6 we defined the upper bound for a problem to be the upper boundof the best algorithm we know for that problem, and the lower bound to be thetightest lower bound that we can prove over all algorithms for that problem. Whilewe usually can recognize the upper bound for a given algorithm, finding the tightestlower bound for all possible algorithms is often difficult, especially if that lowerbound is more than the “trivial” lower bound determined by measuring the amountof input that must be processed.The benefits of being able to discover a strong lower bound are significant. Inparticular, when we can make the upper and lower bounds for a problem meet, thismeans that we truly understand our problem in a theoretical sense. It also savesus the effort of attempting to discover more (asymptotically) efficient algorithmswhen no such algorithm can exist.Often the most effective way to determine the lower bound for a problem isto find a reduction to another problem whose lower bound is already known. Thisis the subject of Chapter 17. However, this approach does not help us when wecannot find a suitable “similar problem.” Our focus in this chapter is discoveringand proving lower bounds from first principles. Our most significant example ofa lower bounds argument so far is the proof from Section 7.9 that the problem ofsorting is O(n log n) in the worst case.Section 15.1 reviews the concept of a lower bound for a problem and presentsthe basic “algorithm” for finding a good algorithm. Section 15.2 discusses lowerbounds on searching in lists, both those that are unordered and those that are ordered.Section 15.3 deals with finding the maximum value in a list, and presents amodel for selection based on building a partially ordered set. Section 15.4 presents485
486 Chap. 15 Lower Boundsthe concept of an adversarial lower bounds proof. Section 15.5 illustrates the conceptof a state space lower bound. Section 15.6 presents a linear time worst-casealgorithm for finding the ith biggest element on a list. Section 15.7 continues ourdiscussion of sorting with a quest for the algorithm that requires the absolute fewestnumber of comparisons needed to sort a list.15.1 Introduction to Lower Bounds ProofsThe lower bound for the problem is the tightest (highest) lower bound that we canprove for all possible algorithms that solve the problem. 1 This can be a difficult bar,given that we cannot possibly know all algorithms for any problem, because thereare theoretically an infinite number. However, we can often recognize a simplelower bound based on the amount of input that must be examined. For example,we can argue that the lower bound for any algorithm to find the maximum-valuedelement in an unsorted list must be Ω(n) because any algorithm must examine allof the inputs to be sure that it actually finds the maximum value.In the case of maximum finding, the fact that we know of a simple algorithmthat runs in O(n) time, combined with the fact that any algorithm needs Ω(n) time,is significant. Because our upper and lower bounds meet (within a constant factor),we know that we do have a “good” algorithm for solving the problem. It is possiblethat someone can develop an implementation that is a “little” faster than an existingone, by a constant factor. But we know that its not possible to develop one that isasymptotically better.We must be careful about how we interpret this last statement, however. Theworld is certainly better off for the invention of Quicksort, even though Mergesortwas available at the time. Quicksort is not asymptotically faster than Mergesort, yetis not merely a “tuning” of Mergesort either. Quicksort is a substantially differentapproach to sorting. So even when our upper and lower bounds for a problem meet,there are still benefits to be gained from a new, clever algorithm.So now we have an answer to the question “How do I know if I have a goodalgorithm to solve a problem?” An algorithm is good (asymptotically speaking) ifits upper bound matches the problem’s lower bound. If they match, we know tostop trying to find an (asymptotically) faster algorithm. What if the (known) upperbound for our algorithm does not match the (known) lower bound for the problem?In this case, we might not know what to do. Is our upper bound flawed, and thealgorithm is really faster than we can prove? Is our lower bound weak, and the truelower bound for the problem is greater? Or is our algorithm simply not the best?1 Throughout this discussion, it should be understood that any mention of bounds must specifywhat class of inputs are being considered. Do we mean the bound for the worst case input? Theaverage cost over all inputs? Regardless of which class of inputs we consider, all of the issues raisedapply equally.
Page 1 and 2:
Data Structures and AlgorithmAnalys
Page 3 and 4:
ivContents2.6.1 Direct Proof 372.6.
Page 5 and 6:
viContents6.1 General Tree Definiti
Page 7 and 8:
viiiContents10.5.2 B-Tree Analysis
Page 9 and 10:
xContents15.7 Optimal Sorting 50115
Page 12 and 13:
PrefaceWe study data structures so
Page 14 and 15:
PrefacexvA sophomore-level class wh
Page 16 and 17:
Prefacexviithe form ofcalls to meth
Page 18:
PrefacexixFor the second edition, I
Page 22 and 23:
1Data Structures and AlgorithmsHow
Page 24 and 25:
Sec. 1.1 A Philosophy of Data Struc
Page 26 and 27:
Sec. 1.1 A Philosophy of Data Struc
Page 28 and 29:
Sec. 1.2 Abstract Data Types and Da
Page 30 and 31:
Sec. 1.2 Abstract Data Types and Da
Page 32 and 33:
Sec. 1.3 Design Patterns 13that tra
Page 34 and 35:
Sec. 1.3 Design Patterns 15will cal
Page 36 and 37:
Sec. 1.4 Problems, Algorithms, and
Page 38 and 39:
Sec. 1.5 Further Reading 19vides po
Page 40 and 41:
Sec. 1.6 Exercises 21than 10,000 of
Page 42 and 43:
2Mathematical PreliminariesThis cha
Page 44 and 45:
Sec. 2.1 Sets and Relations 25A seq
Page 46 and 47:
Sec. 2.2 Miscellaneous Notation 27E
Page 48 and 49:
Sec. 2.3 Logarithms 29Unfortunately
Page 50 and 51:
Sec. 2.4 Summations and Recurrences
Page 52 and 53:
Sec. 2.4 Summations and Recurrences
Page 54 and 55:
Sec. 2.5 Recursion 35(a)(b)Figure 2
Page 56 and 57:
Sec. 2.6 Mathematical Proof Techniq
Page 58 and 59:
Page 60 and 61:
Page 62 and 63:
Page 64 and 65:
Sec. 2.8 Further Reading 45one inch
Page 66 and 67:
Sec. 2.9 Exercises 47(c) For nonzer
Page 68 and 69:
Sec. 2.9 Exercises 492.16 Write a r
Page 70 and 71:
Sec. 2.9 Exercises 512.38 How many
Page 72 and 73:
3Algorithm AnalysisHow long will it
Page 74 and 75:
Sec. 3.1 Introduction 55efficient.
Page 76 and 77:
Sec. 3.1 Introduction 57n! 2 n2n 25
Page 78 and 79:
Sec. 3.2 Best, Worst, and Average C
Page 80 and 81:
Sec. 3.3 A Faster Computer, or a Fa
Page 82 and 83:
Sec. 3.4 Asymptotic Analysis 633.4
Page 84 and 85:
Sec. 3.4 Asymptotic Analysis 65If s
Page 86 and 87:
Sec. 3.4 Asymptotic Analysis 67it i
Page 88 and 89:
Sec. 3.5 Calculating the Running Ti
Page 90 and 91:
Page 92 and 93:
Page 94 and 95:
Sec. 3.7 Common Misunderstandings 7
Page 96 and 97:
Sec. 3.8 Multiple Parameters 77the
Page 98 and 99:
Sec. 3.9 Space Bounds 79A data stru
Page 100 and 101:
Sec. 3.10 Speeding Up Your Programs
Page 102 and 103:
Sec. 3.11 Empirical Analysis 833.11
Page 104 and 105:
Sec. 3.13 Exercises 853.13 Exercise
Page 106 and 107:
Sec. 3.13 Exercises 87(f) sum = 0;f
Page 108:
Sec. 3.14 Projects 893.24 Prove tha
Page 112 and 113:
4Lists, Stacks, and QueuesIf your p
Page 114 and 115:
Sec. 4.1 Lists 95list ADT. Interfac
Page 116 and 117:
Sec. 4.1 Lists 97for (L.moveToStart
Page 118 and 119:
Sec. 4.1 Lists 99public void moveTo
Page 120 and 121:
Sec. 4.1 Lists 101/** Singly linked
Page 122 and 123:
Sec. 4.1 Lists 103headcurrtailhead2
Page 124 and 125:
Sec. 4.1 Lists 105/** Set curr at l
Page 126 and 127:
Sec. 4.1 Lists 107curr...23 10 12(a
Page 128 and 129:
Sec. 4.1 Lists 109/** Singly linked
Page 130 and 131:
Sec. 4.1 Lists 111Array-based lists
Page 132 and 133:
Sec. 4.1 Lists 113headcurrtail20 23
Page 134 and 135:
Sec. 4.1 Lists 115/** Insert "it" a
Page 136 and 137:
Sec. 4.2 Stacks 117The principle be
Page 138 and 139:
Sec. 4.2 Stacks 119/** Array-based
Page 140 and 141:
Sec. 4.2 Stacks 121top1top2Figure 4
Page 142 and 143:
Sec. 4.2 Stacks 123you might want t
Page 144 and 145:
Sec. 4.3 Queues 125/** Queue ADT */
Page 146 and 147:
Sec. 4.3 Queues 127frontfront20 512
Page 148 and 149:
Sec. 4.3 Queues 129/** Array-based
Page 150 and 151:
Sec. 4.4 Dictionaries 1314.3.3 Comp
Page 152 and 153:
Sec. 4.4 Dictionaries 133not get at
Page 154 and 155:
Sec. 4.4 Dictionaries 135/** Contai
Page 156 and 157:
Sec. 4.4 Dictionaries 137}/** @retu
Page 158 and 159:
Sec. 4.6 Exercises 139(b) L2.append
Page 160 and 161:
Sec. 4.7 Projects 1414.16 Re-implem
Page 162:
Sec. 4.7 Projects 143‘a’ ‘b
Page 165 and 166:
146 Chap. 5 Binary TreesABCDEFGHIFi
Page 167 and 168:
148 Chap. 5 Binary TreesAny number
Page 169 and 170:
150 Chap. 5 Binary Trees/** ADT for
Page 171 and 172:
152 Chap. 5 Binary Treestends to be
Page 173 and 174:
154 Chap. 5 Binary Trees5.3 Binary
Page 175 and 176:
156 Chap. 5 Binary TreesABCDEFGHIFi
Page 177 and 178:
158 Chap. 5 Binary Treesto its call
Page 179 and 180:
160 Chap. 5 Binary Trees5.3.2 Space
Page 181 and 182:
162 Chap. 5 Binary Trees012345 678
Page 183 and 184:
164 Chap. 5 Binary Trees12037422442
Page 185 and 186:
166 Chap. 5 Binary Trees37244273240
Page 187 and 188:
168 Chap. 5 Binary Treessubroot105
Page 189 and 190:
170 Chap. 5 Binary Treesin the case
Page 191 and 192:
172 Chap. 5 Binary Treessynonymous
Page 193 and 194:
174 Chap. 5 Binary Trees/** Heapify
Page 195 and 196:
176 Chap. 5 Binary TreesRH1H2Figure
Page 197 and 198:
178 Chap. 5 Binary Trees5.6 Huffman
Page 199 and 200:
180 Chap. 5 Binary TreesLetter C D
Page 201 and 202:
182 Chap. 5 Binary Trees/** Huffman
Page 203 and 204:
184 Chap. 5 Binary Trees/** Build a
Page 205 and 206:
186 Chap. 5 Binary Treesa reverse p
Page 207 and 208:
188 Chap. 5 Binary Trees18 bits, mo
Page 209 and 210:
190 Chap. 5 Binary Trees5.12 Write
Page 211 and 212:
192 Chap. 5 Binary Trees(c) What is
Page 213 and 214:
194 Chap. 5 Binary Trees5.7 Complet
Page 215 and 216:
196 Chap. 6 Non-Binary TreesRootRAn
Page 217 and 218:
198 Chap. 6 Non-Binary TreesRABCD E
Page 219 and 220:
200 Chap. 6 Non-Binary Trees/** Gen
Page 221 and 222:
202 Chap. 6 Non-Binary Treesspond t
Page 223 and 224:
204 Chap. 6 Non-Binary Treesditiona
Page 225 and 226:
206 Chap. 6 Non-Binary Treeslence o
Page 227 and 228:
208 Chap. 6 Non-Binary TreesR’RA
Page 229 and 230:
210 Chap. 6 Non-Binary TreesRRABABC
Page 231 and 232:
212 Chap. 6 Non-Binary Trees(a)(b)F
Page 233 and 234:
214 Chap. 6 Non-Binary Treestwo chi
Page 235 and 236:
216 Chap. 6 Non-Binary Treeschild v
Page 237 and 238:
218 Chap. 6 Non-Binary TreesCAFBEHG
Page 239 and 240:
220 Chap. 6 Non-Binary Treestree af
Page 242 and 243:
7Internal SortingWe sort many thing
Page 244 and 245:
Sec. 7.2 Three Θ(n 2 ) Sorting Alg
Page 246 and 247:
Page 248 and 249:
Page 250 and 251:
Sec. 7.3 Shellsort 231Insertion Bub
Page 252 and 253:
Sec. 7.4 Mergesort 233static
Page 254 and 255:
Sec. 7.4 Mergesort 235static
Page 256 and 257:
Sec. 7.5 Quicksort 237Quicksort is
Page 258 and 259:
Sec. 7.5 Quicksort 239static
Page 260 and 261:
Sec. 7.5 Quicksort 241is still unli
Page 262 and 263:
Sec. 7.6 Heapsort 243subarray, only
Page 264 and 265:
Sec. 7.7 Binsort and Radix Sort 245
Page 266 and 267:
Page 268 and 269:
Page 270 and 271:
Sec. 7.8 An Empirical Comparison of
Page 272 and 273:
Sec. 7.9 Lower Bounds for Sorting 2
Page 274 and 275:
Sec. 7.9 Lower Bounds for Sorting 2
Page 276 and 277:
Sec. 7.10 Further Reading 257• Eq
Page 278 and 279:
Sec. 7.11 Exercises 259algorithm to
Page 280 and 281:
Sec. 7.12 Projects 2617.18 Which of
Page 282:
Sec. 7.12 Projects 2637.6 It has be
Page 285 and 286:
266 Chap. 8 File Processing and Ext
Page 287 and 288:
Page 289 and 290:
Page 291 and 292:
Page 293 and 294:
Page 295 and 296:
Page 297 and 298:
Page 299 and 300:
Page 301 and 302:
Page 303 and 304:
Page 305 and 306:
Page 307 and 308:
Page 309 and 310:
Page 311 and 312:
Page 313 and 314:
Page 315 and 316:
Page 317 and 318:
Page 319 and 320:
Page 321 and 322:
302 Chap. 9 Searchingintroduced in
Page 323 and 324:
304 Chap. 9 Searchingvalue in L gre
Page 325 and 326:
306 Chap. 9 SearchingWe now show th
Page 327 and 328:
308 Chap. 9 Searchingrecords by exp
Page 329 and 330:
310 Chap. 9 SearchingThis is potent
Page 331 and 332:
312 Chap. 9 Searchingand the total
Page 333 and 334:
314 Chap. 9 SearchingThis method of
Page 335 and 336:
316 Chap. 9 Searchingbirthday is si
Page 337 and 338:
318 Chap. 9 Searching45674567319692
Page 339 and 340:
320 Chap. 9 Searching01000953012330
Page 341 and 342:
322 Chap. 9 Searching01HashTable100
Page 343 and 344:
324 Chap. 9 Searching/** Insert rec
Page 345 and 346:
326 Chap. 9 Searching09050090501100
Page 347 and 348:
328 Chap. 9 SearchingExample 9.9 Co
Page 349 and 350:
330 Chap. 9 Searching/** Dictionary
Page 351 and 352:
332 Chap. 9 SearchingThe cost for a
Page 353 and 354:
334 Chap. 9 Searchingthat is not in
Page 355 and 356:
336 Chap. 9 SearchingA good introdu
Page 357 and 358:
338 Chap. 9 Searching9.16 Assume th
Page 360 and 361:
10IndexingMany large-scale computin
Page 362 and 363:
Sec. 10.1 Linear Indexing 343Linear
Page 364 and 365:
Sec. 10.1 Linear Indexing 345JonesA
Page 366 and 367:
Sec. 10.2 ISAM 347In−memoryTable
Page 368 and 369:
Sec. 10.3 Tree-based Indexing 349Fi
Page 370 and 371:
Sec. 10.4 2-3 Trees 351/** 2-3 tree
Page 372 and 373:
Sec. 10.4 2-3 Trees 35318331223 30
Page 374 and 375:
Sec. 10.5 B-Trees 355private TTNode
Page 376 and 377:
Sec. 10.5 B-Trees 35724152033 45 48
Page 378 and 379:
Sec. 10.5 B-Trees 3593318234810 12
Page 380 and 381:
Sec. 10.5 B-Trees 361331012 233348
Page 382 and 383:
Sec. 10.5 B-Trees 3634845 4748 50 5
Page 384 and 385:
Sec. 10.6 Further Reading 365at mos
Page 386 and 387:
Sec. 10.8 Projects 36710.7 Prove th
Page 388:
PART IVAdvanced Data Structures369
Page 391 and 392:
372 Chap. 11 Graphs04123147123(a)(b
Page 393 and 394:
374 Chap. 11 Graphs0 1 2 3 40201 11
Page 395 and 396:
376 Chap. 11 Graphstime. This is a
Page 397 and 398:
378 Chap. 11 GraphsIt is reasonably
Page 399 and 400:
380 Chap. 11 Graphs}/** @return an
Page 401 and 402:
382 Chap. 11 Graphs}/** Set the wei
Page 403 and 404:
384 Chap. 11 GraphsABABCCDDFFEE(a)(
Page 405 and 406:
386 Chap. 11 Graphs/** Breadth firs
Page 407 and 408:
388 Chap. 11 Graphs/** Recursive to
Page 409 and 410:
390 Chap. 11 GraphsA103B2205D11C15E
Page 411 and 412:
392 Chap. 11 Graphs/** Dijkstra’s
Page 413 and 414:
394 Chap. 11 GraphsA7 5CB91 26ED 21
Page 415 and 416:
396 Chap. 11 GraphsPrim’s algorit
Page 417 and 418:
398 Chap. 11 GraphsInitialA B C D E
Page 419 and 420:
400 Chap. 11 Graphs(a) IF an undire
Page 421 and 422:
402 Chap. 11 Graphs11.8 Projects11.
Page 424 and 425:
12Lists and Arrays RevisitedSimple
Page 426 and 427:
Sec. 12.1 Multilists 407L1L2L3bcdL4
Page 428 and 429:
Sec. 12.2 Matrix Representations 40
Page 430 and 431:
Sec. 12.2 Matrix Representations 41
Page 432 and 433:
Sec. 12.3 Memory Management 413/**
Page 434 and 435:
Sec. 12.3 Memory Management 415Smal
Page 436 and 437:
Sec. 12.3 Memory Management 417Tag
Page 438 and 439:
Sec. 12.3 Memory Management 419tend
Page 440 and 441:
Sec. 12.3 Memory Management 421is e
Page 442 and 443:
Sec. 12.3 Memory Management 423AaBc
Page 444 and 445:
Sec. 12.4 Further Reading 4251a3b42
Page 446 and 447:
Sec. 12.6 Projects 42712.7 Write a
Page 448 and 449:
13Advanced Tree StructuresThis chap
Page 450 and 451:
Sec. 13.1 Tries 4310101010120001240
Page 452 and 453: Sec. 13.1 Tries 433000xxxx00xxxxx0x
Page 454 and 455: Sec. 13.2 Balanced Trees 4353737244
Page 456 and 457: Sec. 13.2 Balanced Trees 437that is
Page 458 and 459: Sec. 13.2 Balanced Trees 439GPDSASP
Page 460 and 461: Sec. 13.3 Spatial Data Structures 4
Page 472 and 473: Sec. 13.4 Further Reading 45313.3.4
Page 474 and 475: Sec. 13.6 Projects 455(c) Show the
Page 476: Sec. 13.6 Projects 45713.11 Select
Page 480 and 481: 14Analysis TechniquesOften it is ea
Page 482 and 483: Sec. 14.1 Summation Techniques 463w
Page 484 and 485: Sec. 14.1 Summation Techniques 465B
Page 486 and 487: Sec. 14.2 Recurrence Relations 4671
Page 488 and 489: Sec. 14.2 Recurrence Relations 469n
Page 490 and 491: Sec. 14.2 Recurrence Relations 471E
Page 492 and 493: Sec. 14.2 Recurrence Relations 473N
Page 494 and 495: Sec. 14.2 Recurrence Relations 475T
Page 496 and 497: Sec. 14.3 Amortized Analysis 477(ch
Page 498 and 499: Sec. 14.4 Further Reading 479compar
Page 500 and 501: Sec. 14.5 Exercises 48114.14 For th
Page 502: Sec. 14.6 Projects 48314.24 Use mat
Page 507 and 508: 488 Chap. 15 Lower Boundsat least o
Page 509 and 510: 490 Chap. 15 Lower BoundsABCGDEFFig
Page 511 and 512: 492 Chap. 15 Lower BoundsProof 1: T
Page 513 and 514: 494 Chap. 15 Lower BoundsFigure 15.
Page 515 and 516: 496 Chap. 15 Lower BoundsTo minimiz
Page 517 and 518: 498 Chap. 15 Lower BoundsThis recur
Page 519 and 520: 500 Chap. 15 Lower BoundsFigure 15.
Page 521 and 522: 502 Chap. 15 Lower Boundsnot only t
Page 523 and 524: 504 Chap. 15 Lower Bounds1234Figure
Page 525 and 526: 506 Chap. 15 Lower Bounds(a) Assume
Page 528 and 529: 16Patterns of AlgorithmsThis chapte
Page 530 and 531: Sec. 16.1 Dynamic Programming 511Dy
Page 532 and 533: Sec. 16.1 Dynamic Programming 513on
Page 534 and 535: Sec. 16.2 Randomized Algorithms 515
Page 542 and 543: Sec. 16.3 Numerical Algorithms 523
Page 544 and 545: Sec. 16.3 Numerical Algorithms 525t
Page 546 and 547: Sec. 16.3 Numerical Algorithms 527B
Page 548 and 549: Sec. 16.3 Numerical Algorithms 529o
Page 550 and 551: Sec. 16.3 Numerical Algorithms 531o
Page 552: Sec. 16.6 Projects 533value depth5
Page 555 and 556:
536 Chap. 17 Limits to ComputationT
Page 557 and 558:
Page 559 and 560:
540 Chap. 17 Limits to ComputationS
Page 561 and 562:
542 Chap. 17 Limits to Computationa
Page 563 and 564:
544 Chap. 17 Limits to Computation3
Page 565 and 566:
Page 567 and 568:
548 Chap. 17 Limits to Computationp
Page 569 and 570:
550 Chap. 17 Limits to ComputationE
Page 571 and 572:
552 Chap. 17 Limits to Computation1
Page 573 and 574:
554 Chap. 17 Limits to ComputationM
Page 575 and 576:
556 Chap. 17 Limits to Computationw
Page 577 and 578:
558 Chap. 17 Limits to Computationx
Page 579 and 580:
560 Chap. 17 Limits to Computationt
Page 581 and 582:
562 Chap. 17 Limits to Computationm
Page 583 and 584:
564 Chap. 17 Limits to Computation(
Page 586 and 587:
Bibliography[AG06] Ken Arnold and J
Page 588 and 589:
BIBLIOGRAPHY 569[GI91] Zvi Galil an
Page 590 and 591:
BIBLIOGRAPHY 571[SJH93] Clifford A.
Page 592 and 593:
Index80/20 rule, 309, 333abstract d
Page 594 and 595:
INDEX 575image space, 430key space,
Page 596 and 597:
INDEX 577building, 172-177for memor
Page 598 and 599:
INDEX 579octree, 451Ω notation, 65
Page 600 and 601:
INDEX 581comparing algorithms, 224-
show all

Data Structures and Algorithm Analysis - Computer Science at ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?