A Practical Introduction to Data Structures and Algorithm Analysis

Sec. 7.9 Lower Bounds for Sorting 267arrays, optimized Quicksort performs well because it does one partition step beforecalling Insertion Sort. Compared to the other O(n log n) sorts, unoptimizedHeapsort is quite slow due to the overhead of the class structure. When all of thisis stripped away and the algorithm is implemented to manipulate an array directly,it is still somewhat slower than mergesort. In general, optimizing the various algorithmsmakes a noticeable improvement for larger array sizes.Overall, Radix Sort is a surprisingly poor performer. If the code had been tunedto use bit shifting of the key value, it would likely improve substantially; but thiswould seriously limit the range of element types that the sort could support.7.9 Lower Bounds for SortingThis book contains many analyses for algorithms. These analyses generally definethe upper and lower bounds for algorithms in their worst and average cases. Formost of the algorithms presented so far, analysis is easy. This section considersa more difficult task — an analysis for the cost of a problem as opposed to analgorithm. The upper bound for a problem can be defined as the asymptotic cost ofthe fastest known algorithm. The lower bound defines the best possible efficiencyfor any algorithm that solves the problem, including algorithms not yet invented.Once the upper and lower bounds for the problem meet, we know that no futurealgorithm can possibly be (asymptotically) more efficient.A simple estimate for a problem’s lower bound can be obtained by measuringthe size of the input that must be read and the output that must be written. Certainlyno algorithm can be more efficient than the problem’s I/O time. From this we seethat the sorting problem cannot be solved by any algorithm in less than Ω(n) timebecause it takes at least n steps to read and write the n values to be sorted. Basedon our current knowledge of sorting algorithms and the size of the input, we knowthat the problem of sorting is bounded by Ω(n) and O(n log n).Computer scientists have spent much time devising efficient general-purposesorting algorithms, but no one has ever found one that is faster than O(n log n) inthe worst or average cases. Should we keep searching for a faster sorting algorithm?Or can we prove that there is no faster sorting algorithm by finding a tighter lowerbound?This section presents one of the most important and most useful proofs in computerscience: No sorting algorithm based on key comparisons can possibly befaster than Ω(n log n) in the worst case. This proof is important for three reasons.First, knowing that widely used sorting algorithms are asymptotically optimal is reassuring.In particular, it means that you need not bang your head against the wallsearching for an O(n) sorting algorithm (or at least not one in any way based on key