A Practical Introduction to Data Structures and Algorithm Analysis

560 Chap. 17 Limits to Computationslow, but perhaps there are better ones waiting to be discovered. Of course, whilehaving a problem with high running time is bad, it is even worse to have a problemthat cannot be solved at all! Problems of the later type do exist, and some arepresented in Section 17.3.This chapter presents a brief introduction to the theory of expensive and impossibleproblems. Section 17.1 presents the concept of a reduction, which is thecentral tool used for analyzing the difficulty of a problem (as opposed to analyzingthe cost of an algorithm). Reductions allow us to relate the difficulty of variousproblems, which is often much easier than doing the analysis for a problem fromfirst principles. Section 17.2 discusses “hard” problems, by which we mean problemsthat require, or at least appear to require, time exponential on the input size.Finally, Section 17.3 considers various problems that, while often simple to defineand comprehend, are in fact impossible to solve using a computer program. Theclassic example of such a problem is deciding whether an arbitrary computer programwill go into an infinite loop when processing a specified input. This is knownas the halting problem.17.1 ReductionsWe begin with an important concept for understanding the relationships betweenproblems, called reduction. Reduction allows us to solve one problem in termsof another. Equally importantly, when we wish to understand the difficulty of aproblem, reduction allows us to make relative statements about upper and lowerbounds on the cost of a problem (as opposed to an algorithm or program).Because the concept of a problem is discussed extensively in this chapter, wewant notation to simplify problem descriptions. Throughout this chapter, a problemwill be defined in terms of a mapping between inputs and outputs, and the name ofthe problem will be given in all capital letters. Thus, a complete definition of thesorting problem could appear as follows:SORTING:Input: A sequence of integers x 0 , x 1 , x 2 , ..., x n−1 .Output: A permutation y 0 , y 1 , y 2 , ..., y n−1 of the sequence such that y i ≤ y jwhenever i < j.Once you have bought or written a program to solve one problem, such assorting, you might be able to use it as a tool to solve a different problem. This is