CSE 2320 Notes 2: Growth of Functions f(n) g(n) cg(n)

CSE 2320 Notes 2: Growth of Functions
1
(Last updated 12/26/13 12:39 PM)
Sedgewick 2.1-2.3
Why constants are annoying . . .
Problem Instance of Size n (+ Constraints)
⇓
Algorithm for Problem
⇓
Language
⇓
Compiler
⇓
Machine
clock
memory levels
word length
instruction set
⇓
Resources Consumed (time, memory, processors, bandwidth)
Need to compare time and space usage of various algorithms for a problem.
2.A. ASYMPTOTIC NOTATION
Sedgewick 2.4
Ο( g( n)
) is a set of functions:
( ) ∈ Ο g( n)
f n
( ) iff ∃ c and
n 0 such that
0 ≤ f ( n) ≤ cg( n) when
n ≥ n 0
€
cg(n)
€
€
€
€
f(n)
g(n)
Theorem: If
lim
n→∞
f (n)
g(n)
is a constant, then f(n) ∈ O(g(n)).
€

f(n) = n 2 , g(n) = n 3
f(n) = n 3 , g(n) = n 2
€
€
lim n2
n→∞ n 3 = lim
n→∞
f(n) = 3n 2 + 2n – 3, g(n) = 5n 2 – n + 2
lim 3n2 + 2n − 3
n→∞ 5n 2 =
− n + 2
2n
3n 2 = lim 2
n→∞ 6n = 0 ⇒ n2 ∈ Ο(n 3 )
lim n3
n→∞ n 2 = lim 3n2
n→∞ 2n = lim 6n
n→∞ 2 = unbounded ⇒ n3 ∉ Ο(n 2 )
lim 6n + 2
n→∞
10n −1 = lim 6
n→∞10 = 3 5 ⇒ 3n2 + 2n − 3 ∈ Ο(5n 2 − n + 2)
2
Conclusion: Toss out low-order terms
€
Is sin( x) ∈ Ο( cos( x)
)
€
n k ⎛
∈ Ο⎜
n l ⎞
⎟ if
⎝ ⎠
€
l ≥ k.
cos(x)
€
sin(x)
Is
( ) ∈ Ο log 10 ( x)
ln x
( )
€
€
Is 2 n ⎛
∈ Ο⎜
n k ⎞
⎟ for some fixed k
⎝ ⎠
n k ⎛
∈ Ο⎜
2 n ⎞
⎟ for any k. General case, n k ⎛
∈ Ο⎜
c n ⎞
⎟ for any c > 1
⎝ ⎠
⎝ ⎠
Be sure to appreciate Tables 2.1 and 2.2 in Sedgewick
€
( ) is a set of functions:
Ω g( n)
€
( ) ∈ Ω g( n)
f n
( ) iff ∃ c and
n0 such that
0 ≤ cg( n) ≤ f ( n) ( c > 0) when
n ≥ n0
€
g(n)
€
€
€
€
f(n)
cg(n)

Theorem: If
lim g(n)
n→∞ f (n)
is a constant, then
( ) ∈ Ω g( n)
f n
( ).
3
( ) = n 2 ,
g n
€
f ( n) = 3n 3
lim
n→∞
n2 €
3n 3 = lim
2n
n→∞ 9n 2 = lim 2
n→∞18n = 0 ⇒ n3 ∈ Ω(n 2 )
€
Upper bounds (O()) are for particular algorithms.
€
Lower bounds € (Ω()) are for indicating the inherent difficulty of problems.
The hunt for an optimal algorithm . . . (Sedgewick 2.7)
http://rjlipton.wordpress.com/2012/02/01/a-brief-history-of-matrix-product/
Best upper bound:
⎛
Ο⎜
n 2.373 ⎞
⎟
⎝ ⎠
Best lower bound:
⎛
Ω⎜
n 2 ⎞
⎟
⎝ ⎠
Θ-notation (asymptotically tight bound) – Used to express time for worst-case instances for algorithm
f
€
n
( ) iff
( ) ∈ Θ g( n)
( ) ∈ Ο g( n)
f n
( ) and
f n
€
( ).
( ) ∈ Ω g( n)
Theorem: If
€
lim g(n)
n→∞ f (n)
€
is a constant > 0, then f(n) ∈ Θ(g(n)).
€
f(n) = 3n 2 + 2n – 3, g(n) = 5n 2 + n - 1
€
lim 3n2 + 2n − 3
n→∞ 5n 2 =
+ n −1
lim 6n + 2
n→∞
10n +1 = lim 6
n→∞10 = 3 5 ⇒ 3n2 + 2n − 3 ∈ Θ(5n 2 + n −1)
€
€
Θ( f ( n)
) is an equivalence relation . . .
Symmetric:
Reflexive:
Transitive:
( ) ∈ Θ f ( n)
( ) ∈ Θ f ( n)
( ) ∈ Θ g( n)
g n
f n
f n
€
€
Accepted
€
abuses of asymptotic notation.
€
€
1. n 2 ⎛
+ n = Θ⎜
n 2 ⎞
⎟ means
⎝ ⎠
( ) ⇔ f ( n) ∈ Θ( g( n)
) [Doesn’t work for O or Ω]
( ) [Works for O and Ω]
( ) and g( n) ∈ Θ( h( n)
) ⇒ f ( n) ∈ Θ( h( n)
) [Works for O and Ω]
€
n 2 ⎛
+ n ∈ Θ⎜
n 2 ⎞
⎟ .
⎝ ⎠
€
€
€
2.
3.
3n 2 + 2n +1= 3n 2 + Θ n
2 + 3Θ n
€
( ) means . . .
( ) = n + Θ( n) means . . .

Aside: Indicating that a bound could be improved:
4
o(f) = O(f) - Θ(f)
n 2 ∈ o(n 3 )
ω(f) = Ω(f) - Θ(f)
1
n 2 ∈ ω ⎛ 1 ⎞
⎜
⎝ n 3 ⎟
⎠
Example: p. 46 of Demaine & O’Rourke, Geometric Folding Algorithms: Linkages, Origami, &
Polyhedra, Cambridge Univ. Press, 2007.
€
⎛
“Open Problem 4.1: Faster Generic Rigidity in 2D. Is there a o⎜
n 2 ⎞
⎟ -time algorithm to test generic
⎝ ⎠
rigidity of a given graph with n vertices” (http://minerva.cs.mtholyoke.edu/demos)
2.B. APPLYING ASYMPTOTIC ANALYSIS TO CODE SEGMENTS
Sedgewick 2.6
for (i=0; i

equalCount=0;
for (i=0; i

c=1.0/ab[k][k];
for (i=k+1;i

2.C. DEMONSTRATING FUNDAMENTAL RESULTS FOR ASYMPTOTIC NOTATION
7
f(n) and g(n) are positive functions. Show max(f(n),g(n)) = Θ(f(n) + g(n))
O: Since f(n) ≤ f(n) + g(n) and g(n) ≤ f(n) + g(n)
max(f(n),g(n)) ≤ f(n) + g(n) for all n, so n 0 = 0 and c = 1
Ω: Must have c(f(n) + g(n)) ≤ max(f(n),g(n)), so choose n 0 = 0 and c ≤ 1/2 [Consider f(n) = g(n)]
_c_ _+_ _f_ _g_ _max_
10 1 9 9
10 2 8 8
10 3 7 7
10 5 5 5
Test Problem: Prove that if
( ) ∈ Ω f ( n)
g n
( ) then
( ) ∈ Ο g( n)
f n
( ).
cf ( n) ≤ g( n) when n ≥ n 0
So, f ( n) ≤ € 1 c g n
( ) when n ≥ n 0
Need f ( n) ≤ dg( n) when n ≥ n 1
€
Choose d = 1 c and n 1 = n 0
2.D. MISCELLANEOUS
€ floor(x) =
⎣x⎦
Largest integer ≤ x
⎣2.5⎦ =
⎣−2.5⎦ =
ceiling(x) =
€
€
Worst-case number of probes for binary search:
€
⎡x⎤
Smallest integer ≥ x
⎡2.5⎤ =
€
⎣lg n⎦+1
⎡−2.5⎤ =
Consider worst-case searches on tables
€
with 15 elements
€
(4) and 16 elements (5).
Review:
2 n ⎛
Ο⎜
3 n ⎞
⎟
⎝ ⎠
2 n ⎛
Ω⎜
3 n ⎞
⎟
⎝ ⎠
€ ⎛
lg⎜
2 n ⎞
⎟ = n
⎝ ⎠
⎛
lg⎜
3 n ⎞
⎟ = nlg3
⎝ ⎠
⎛
lg⎜
2 n ⎞
⎝ ⎠
⎟ = Θ ⎛
lg ⎛
⎜ 3 n ⎞ ⎞
⎜ ⎟ ⎟
⎝ ⎝ ⎠ ⎠
Factorials:
€
n
n!= ∏ i =1•2• 3•...• n
i=1
€

( ) = lgi
lg n!
n
∑
i=1
∈ Θ( n lgn)
8
€
n n! lg(n!) n lg n
1 1 0 0
2 2 1 2
3 6 2.58 4.75
4 24 4.58 8
5 120 6.91 11.61
6 720 9.49 15.51
7 5040 12.3 19.65
8 40320 15.3 24
9 362880 18.47 28.53
10 3628800 21.79 33.22
11 39916800 25.25 38.05
12 479001600 28.84 43.02
13 6227020800 32.54 48.11
14 87178291200 36.34 53.30
15 1307674368000 40.25 58.60
20 2.43E18 61.08 86.44
200 1245 1528
2000 19053 21932
20000 256909 285754
200000 3233399 3521928
2000000 38977759 41863137
€
O:
€ Ω:
€
€
€
€
n!≤ n n
lg n!
lg n!
( ) ≤ n lg n
( ) ∈ Ο( nlg n)
n!=1•...• n 2 • ⎛
⎜
n ⎝ 2 +1 ⎞
⎠
⎛
⎜
n ⎝ 2 + 2 ⎞
⎟ •...• n
⎠
≥1•...• n n
2 •( n ) 2
n
+ k ≥
2
n for n > 3 and k ≥ 0
n
≥ ( n) 2
1•...• n 2 ≥1
n €
lg(n!) ≥ lg( n ) 2 = n lgn
2 2 = 1 4 nlg n
€
( http://en.wikipedia.org/wiki/Stirling's_approximation )
2.E. ASYMPTOTIC
€
NOTATION FOR TWO PARAMETERS (aside, from Cormen)
( ) =
Ο g( n,m)
( ) : ∃positive constants c,n0, and m0 s.t.
for every ( n,m) pair with either n ≥ n 0 or m ≥ m 0
0 ≤ f ( n,m) ≤ cg( n,m)
⎧ f n,m
⎪
⎨
⎪
⎩
⎫
⎪
⎬
⎪
⎭
€