Data Structures and Algorithm Analysis - Computer Science at ...

64 Chap. 3 Algorithm AnalysisBecause the phrase “has an upper bound to its growth rate of f(n)” is long andoften used when discussing algorithms, we adopt a special notation, called big-Ohnotation. If the upper bound for an algorithm’s growth rate (for, say, the worstcase) is f(n), then we would write that this algorithm is “in the set O(f(n))in theworst case” (or just “in O(f(n))in the worst case”). For example, if n 2 grows asfast as T(n) (the running time of our algorithm) for the worst-case input, we wouldsay the algorithm is “in O(n 2 ) in the worst case.”The following is a precise definition for an upper bound. T(n) represents thetrue running time of the algorithm. f(n) is some expression for the upper bound.For T(n) a non-negatively valued function, T(n) is in set O(f(n))if there exist two positive constants c and n 0 such that T(n) ≤ cf(n)for all n > n 0 .Constant n 0 is the smallest value of n for which the claim of an upper bound holdstrue. Usually n 0 is small, such as 1, but does not need to be. You must also beable to pick some constant c, but it is irrelevant what the value for c actually is.In other words, the definition says that for all inputs of the type in question (suchas the worst case for all inputs of size n) that are large enough (i.e., n > n 0 ), thealgorithm always executes in less than cf(n) steps for some constant c.Example 3.4 Consider the sequential search algorithm for finding a specifiedvalue in an array of integers. If visiting and examining one value inthe array requires c s steps where c s is a positive number, and if the valuewe search for has equal probability of appearing in any position in the array,then in the average case T(n) = c s n/2. For all values of n > 1,c s n/2 ≤ c s n. Therefore, by the definition, T(n) is in O(n) for n 0 = 1 andc = c s .Example 3.5 For a particular algorithm, T(n) = c 1 n 2 + c 2 n in the averagecase where c 1 and c 2 are positive numbers. Then, c 1 n 2 + c 2 n ≤c 1 n 2 + c 2 n 2 ≤ (c 1 + c 2 )n 2 for all n > 1. So, T(n) ≤ cn 2 for c = c 1 + c 2 ,and n 0 = 1. Therefore, T(n) is in O(n 2 ) by the second definition.Example 3.6 Assigning the value from the first position of an array toa variable takes constant time regardless of the size of the array. Thus,T(n) = c (for the best, worst, and average cases). We could say in thiscase that T(n) is in O(c). However, it is traditional to say that an algorithmwhose running time has a constant upper bound is in O(1).