A Practical Introduction to Data Structures and Algorithm Analysis

Sec. 17.2 Hard Problems 567The Towers of Hanoi problem takes exponential time, that is, its running timeis Θ(2 n ). This is radically different from an algorithm that takes Θ(n log n) timeor Θ(n 2 ) time. It is even radically different from a problem that takes Θ(n 4 ) time.These are all examples of polynomial running time, because the exponents for allterms of these equations are constants. Recall from Chapter 3 that if we buy a newcomputer that runs twice as fast, the size of problem with complexity Θ(n 4 ) thatwe can solve in a certain amount of time is increased by the fourth root of two.In other words, there is a multiplicative factor increase, even if it is a rather smallone. This is true for any algorithm whose running time can be represented by apolynomial.Consider what happens if you buy a computer that is twice as fast and try tosolve a bigger Towers of Hanoi problem in a given amount of time. Because itscomplexity is Θ(2 n ), we can solve a problem only one disk bigger! There is nomultiplicative factor, and this is true for any exponential algorithm: A constantfactor increase in processing power results in only a fixed addition in problemsolvingpower.There are a number of other fundamental differences between polynomial runningtimes and exponential running times that argues for treating them as qualitativelydifferent. Polynomials are closed under composition and addition. Thus,running polynomial-time programs in sequence, or having one program with polynomialrunning time call another a polynomial number of times yields polynomialtime. Also, all computers known are polynomially related. That is, any programthat runs in polynomial time on any computer today, when transferred to any othercomputer, will still run in polynomial time.There is a practical reason for recognizing a distinction. In practice, most polynomialtime algorithms are “feasible” in that they can run reasonably large inputsin reasonable time. In contrast, most algorithms requiring exponential time arenot practical to run even for fairly modest sizes of input. One could argue thata program with high polynomial degree (such as n 100 ) is not practical, while anexponential-time program with cost 1.001 n is practical. But the reality is that weknow of almost no problems where the best polynomial-time algorithm has highdegree (they nearly all have degree four or less), while almost no exponential-timealgorithms (whose cost is (O(c n )) have their constant c close to one. So there is notmuch gray area between polynomial and exponential time algorithms in practice.For the rest of this chapter, we define a hard algorithm to be one that runs inexponential time, that is, in Ω(c n ) for some constant c > 1. A definition for a hardproblem will be presented in the next section.