Asymptotic Notation for Functions 1 Big O Notation

Asymptotic Notation for FunctionsCOMP 3200, Discrete StructuresWinter 20061 Big O NotationThe ”big O” notation defined in the following definition is frequently used inthe analysis of computer algorithms.Definition 1.1 Let f(n) and g(n) be functions of an integer variable n. Thenotationf(n) =O(g(n))means there is a real constant c>0 and an integer n 0 such that 1|f(n)| ≤c ·|g(n)| for all n ≥ n 0 . (1)In this situation, c·|g(n)| is called an asymptotic bound of the function |f(n)|.In other words, the notation f(n) =O(g(n)) states that the function |f(n)|is bounded above by a constant multiple of the function |g(n)| for all sufficientlylarge values of n indicated by n ≥ n 0 .Example 1.1 Let f(n) =5n + 6 and g(n) =n 2 . A graph of the two functionsis shown in shown in Fig. 1. Notice that for all n ≥ 6, the straight line f(n) isbounded above by the parabola g(n). This suggests that we can take c = 1 andn 0 =6. In fact, we can show using mathematical induction that5 n +6≤ 1 · n 2 , for all n ≥ 6 (2)(see Exercise 1-(a)).Keep in mind that the values of c and n 0 are not unique. For example, wealso can show that for c = 2 and n 0 =4,5 n +6≤ 2 · n 2 , for all n ≥ 4, (3)(see Exercise 1-(b)).Example 1.2 Consider the functions f(n) =n 3 − n 2 and g(n) =n 3 . First,n 3 − n 2 ≤ n 3 for n ≥ 0 and thereforen 3 − n 2 = O(n 3 ).On the other hand, for n ≥ 2, we haven 3 ≤ n 3 + n 2 (n − 2) = 2 (n 3 − n 2 ),and son 3 = O(n 3 − n 2 ).1 Although we have used absolute value symbols in (1), in most situations the functionsinvolved are positive except possibly for a few initial values of n. In addition, for most casesof interest the variable n ≥ 0.1

Figure 1: The functions f(n) =5n + 6 and g(n) =n 2 .Polynomials and Exponential FunctionsExample 1.3 Let f(n) =n 2 and g(n) =2 n . We can show using mathematicalinduction that c = 1 and n 0 = 4. andn 2 ≤ 1 · 2 n , for all n ≥ 4. (4)The last example is an illustration of the following theorem.Theorem 1.1 Let p(n) be a polynomial and consider the exponential functiona n where a>1. Then p(n) =O(a n ).Polynomials and LogarithmsExample 1.4 Let f(n) =log 2 n and g(n) =n. Then, with c = 1 and n 0 =1,log 2 n ≤ 1 · n, for all n ≥ 1. (5)2

To show (5) we exponentiate both sides and then prove the equivalent statement(see Exercise 2).n ≤ 2 n , for all n ≥ 1 (6)The last example is an illustration of the following theorem.Theorem 1.2 Let p(n) be a polynomial with positive degree and consider thefunction log 2 n. Then log 2 n = O(|p(n)|).2 Big Theta NotationDefinition 2.1 Let f(n) and g(n) be functions of an integer variable n. Thenotationf(n) =Θ(g(n))means thatf(n) =O(g(n)) and g(n) =O(f(n)).An equivalent statement of this definition is there are constants A>0 andB>0 such that for all n ≥ n 0A |g(n)| ≤|f(n)| ≤B |g(n)|.Observe that the definition is symmetric in the sense that f(n) =Θ(g(n)) ifand only if g(n) =Θ(f(n)).Big Θ notation indicates that the functions f(n) and g(n) have equivalentgrowth behavior as n →∞.Example 2.1 Consider the functions f(n) = log 2 n + n and g(n) =n. Then,certainly n = O(log 2 n + n). In addition, we have log 2 n + n = O(n) because(see Exercise 4). Therefore, log 2 n + n =Θ(n).log 2 n + n ≤ 2 n, for all n ≥ 1 (7)Example 2.2 Consider the functions f(n) = n 3 − n 2 and g(n) = n 3 . ByExample 1.2 aboven 3 − n 2 =Θ(n 3 ).The last example is an illustration of the following theorem.Theorem 2.1 Suppose that r and s are non-negative integers and r

Proof. First we consider the case c>0. For n ≥ 0 we haven s ≤ n s + cn r ,and thereforen s = O(n s + cn r ). (9)On the other hand, since r0,n s−r +2c ≥ 0.n s 0 there is an integer n 0 (which depends on c)such that|f(n)| ≤c ·|g(n)|, for all n ≥ n 0 . (13)4

There is an important distinction between this definition and Def. 1.1. InDef. 1.1 only one constant c is involved, while in Def. 3.1, equation (13) holdsfor all constants c>0.Def. 3.1 is equivalent to the statementf(n)lim =0. (14)n→∞ g(n)Example 3.1 Consider the functions f(n) =n + 1 and g(n) =n 2 . ThenTherefore,n +1 1limn→∞ n 2 = limn→∞ n + lim 1n→∞ n 2 =0.n +1=o(n 2 ).The notation f(n) =o(g(n)) indicates that the growth rate of f(n)asn →∞is strictly slower then the growth of g(n). In fact, it is impossible for both ofthe statements f(n) =o(g(n) and f(n) =Θ(g(n)) to be true (see Exercise 9).Example 3.2 Consider the functions f(n) =n log 2 n and g(n) =n 2 . Thenn loglim 2 n logn→∞ n 2 = lim 2 n=0.n→∞ nThe last limit follows since as n →∞, log 2 n → more slowly than n →∞(seeFig. 2). (We can show this formally using L’Hospital’s rule from calculus.)Therefore,n log 2 n = o(n 2 ).4 Analysis of AlgorithmsIn analyzing an algorithm, we are often interested in how the algorithm performsfor large data sets with n elements. Asymptotic notation is very useful for thistype of theoretical analysis.Example 4.1 Consider the sorting algorithm for sorting a sequence of n integersstored in the variables a 1 ,a 2 ,...,a n :for i := 1 to n − 1 dofor j := 1 to n − i doif a j >a j+1 thent := a j ;a j := a j+1 ;a j+1 := t;5

10. (a) Suppose that an algorithm requires W (n) =Θ(n) operations for aproblem of size n. If we double the size of the problem to 2 n, bywhat factor does W (n) change?(b) Same question for W (n) =Θ(n 2 ).(c) Same question for W (n) =Θ(n log 2 n).11. Show for b>1, log b n = Θ(log 2 n).12. For each of the following statements, either explain why the statement istrue or give a counterexample to show that the statement is false.(a) If f(n) =O(g(n)) then f(n) =o(g(n).(b) If f(n) =o(g(n)) then f(n) =O(g(n).(c) If f(n) =Θ(g(n)) then f(n) =O(g(n).(d) If f(n) =O(g(n)) then f(n) =Θ(g(n).8