IEOR 269, Spring 2010 Integer Programming and Combinatorial ...

IEOR269 notes, Prof. Hochbaum, 2010 25
15 Complexity analysis
15.1 Measuring quality of an algorithm
Algorithm: One approach is to enumerate the solutions, and select the best one.
Recall that for the assignment problem with 70 people and 70 tasks there are 70! ≈ 2 332.4 solutions.
The existence of an algorithm does not imply the existence of a good algorithm!
To measure the complexity of a particular algorithm, we count the number of operations that are
performed as a function of the ‘input size’. The idea is to consider each elementary operation
(usually defined as a set of simple arithmetic operations such as {+, −, ×, /, ≤}) as having unit
cost, and measure the number of operations (in terms of the size of the input) required to solve a
problem. The goal is to measure the rate, ignoring constants, at which the running time grows as
the size of the input grows; it is an asymptotic analysis.
Complexity analysis is concerned with counting the number of operations that must be performed
in the worst case.
Definition 15.1 (Concrete Complexity of a problem). The complexity of a problem is the complexity
of the algorithm that has the lowest complexity among all algorithms that solve the problem.
15.1.1 Examples
Set Membership - Unsorted list: We can determine if a particular item is in a list of n items
by looking at each member of the list one by one. Thus the number of comparisons needed
to find a member in an unsorted list of length n is n.
Problem: given a real number x, we want to know if x ∈ S.
Algorithm:
1. Compare x to s i
2. Stop if x = s i
3. else if i ← i + 1 < n goto 1 else stop x is not in S
Complexity = n comparisons in the worst case. This is also the concrete complexity of
this problem. Why?
Set Membership - Sorted list: We can determine if a particular item is in a list of n elements
via binary search. The number of comparisons needed to find a member in a sorted list of
length n is proportional to log 2 n.
Problem: given a real number x, we want to know if x ∈ S.
Algorithm:
1. Select s med = ⌊ first+last
2
⌋ and compare to x
2. If s med = x stop
3. If s med < x then S = (s med+1 , . . . , s last ) else S = (s first , . . . , s med−1 )
4. If first < last goto 1 else stop
Complexity: after k th iteration
n
≤ 2, which implies:
2 k−1
n
2 k−1 elements remain. We are done searching for k such that
log 2 n ≤ k
Thus the total number of comparisons is at most log 2 n.
Aside: This binary search algorithm can be used more generally to find the zero in a monotone
increasing and monotone nondecreasing functions.
Matrix Multiplication: The straightforward method for multiplying two n × n matrices takes
n 3 multiplications and n 2 (n − 1) additions. Algorithms with better complexity (though not