4. Divide and conquer - Framingham State University

4. Divide and conquer 

David M. Keil 

Framingham State University 

44. Divide and conquer 

1. Vector algorithms 

2. Tree algorithms 

3. Solutions to geometric problems 

44. Graph algorithms 

5. Decrease by constant factor 

David Keil Analysis of Algorithms 4. Divide and conquer 1/12 1 

David Keil Analysis of Algorithms 1/12 

Reading: Chs. 3-6 

Divide and conquer 

• Break problem into base and recursive cases 

• We “conquer” by defining the recursive case as 

a simpler version of the original problem 

• Often this achieves time of O(lg n) rather than 

O(n), O(n lg n) rather than O(n2 ) 

• Examples: 

- convex hull 

- binary search 

- closest pair 

- Quicksort Q 

- BST search, traversal 

-Mergesort 

- Heapsort 

- searches of graphs 

- topological sort 

- order statistic 

- matrix multiplication 

David Keil Analysis of Algorithms 4. Divide and conquer 1/12 2



Topic objectives 

4a. Write a recurrence that defines the function 

computed by a divide-and-conquer algorithm 

4b. Write and solve a time recurrence for a divide-andconquer 

algorithm 

5c. Explain an instance of the divide-and-conquer 

approach 

5d. Explain and use traversal methods for trees 

55e. EExplain l i breadth-first b d h fi and d depth-first d h fi graph h searches h 

5f. Explain the basic binary-search-tree algorithms 

5g. Explain the basic algorithms on heaps 


Inquiry 

•Is time scalability a good way to 

measure the performance p of computing p g 

systems? 

• How much does studying design 

approaches help us create efficient 

solutions? 



1. Vector algorithms 

• Binary search 

• Quicksort 

• Mergesort 

• Order statistics 


Decision trees for search 

•The decision tree for a search of 

an array must examine each element 

• Searching g an unsorted array, y, only y 

one element is considered per step 

• Hence the tree depth (running time) is O(n) 

• Searching a sorted array, 

considering the middle element 

enables a decision to search only 

the right or left half of the array 

• Hence the decision tree depth is O(lg n) 


David Keil Analysis of Algorithms 1/12


Binary-search (A, first, last) 

if first > last // (i.e., nothing to search) 

return false 

otherwise 

middle ← (first + last) ÷ 2 

if A[middle] matches key 

return true 

otherwise 

if A[middle] > key 

return Bin-search(A, Bin search(A, first, middle - 1, key) 

otherwise 

return Bin-search(A, middle + 1, last, key) 

> Post: returns true iff key is in A [first…last] 


Binary search recurrences 

Bin-srch(A, x) = 

false if |A| = 0 

true if A[⎣|A| ÷2⎦] = x 

Bi Bin-srch(A[0.. h(A[0 ⎣|A| ÷2⎦]) 2⎦]) if A[⎣|A| ÷2⎦] 2⎦] > x 

Bin-srch(A[⎣|A|÷2⎦+1..|A|]) otherwise 

Time: 

TBin-srch (n) = Θ(1) if n ≤ 1 

Θ(1) + 2T 2TBin Bin-srch srch ((n – 1) / 2) otherwise 

Recursive call examines at most half the remaining 

elements 

Solution to recurrence: TBin-srch(n) = Θ(log2n) = Θ(lg n) 


David Keil Analysis of Algorithms 1/12


Solving recurrences: 

logarithmic time 

T(n) = Θ(1) if n = 1 

Θ(1) + T(n / 2) otherwise 

Solution: Θ(lg n), because at each of 

(n / 2) recursive steps the remaining 

time is cut in half 

Example: Binary search 


Logarithms and running time 

• In algorithm analysis, we can ignore the 

base of a logarithm because logbn is 

proportional to logcn for all constants b 

and c 

• Θ(lg n) = Θ(log2n) = Θ(logbn) for any base b 

• LLogarithmic ith i complexity l it is i desirable d i bl 

because the log function grows slowly 




Correctness of Quicksort 


• To prove: For all i and j less than the size of 

array A, i < j implies A[i] ≤ A[j] 

• Base step: p Property p yto be proven p holds 

vacuously for array of size 1, since i = j 

• Induction step (summary): This step must show 

that if Quicksort leaves each pair of array 

elements in correct order for all arrays of size 

n, n,t then e the t esa same eis stue true for o all a arrays a ayso of size s e 

(n + 1). 

• Our argument uses the fact that each partition 

is of size n or smaller, so each gets sorted 


QuickSort(A) 

If |A| > 1 

(A, pvloc) ← Partition(A) 

QuickSort(A[1.. pvloc − 1]) 

QuickSort(A[pvloc + 1 .. |A|]) 

Return A 

Partition rearranges A such that pvloc is the index of 

an element of A, all elements less than A[pvloc] are 

to its left, left and all elements greater are to its right. right 

Divide-and-conquer is used to enable the recursive 

calls to sort arrays half as large at each recursive 

step 



Partition(A) 

pv ← A[1] 

j ← 1 

k ← 1 

for i ← 2to|A| 2 to |A| 

if A[ i ] < pv 

B[ j ] ← A[ i ] 

j ← j + 1 

else 

C[ k ] ← A[ i ] 

k ← k + 1 

i ← i + 1 

A ← B[1 .. j−1 ] + pv + C[1 .. k−1 ] 

return (A, j ) 


This algorithm copies 

all elements of A less 

than pv p into array y B, all 

others to array C, and 

copies B, pv, and C 

back into A. Its running 

time is O(n) 


Partition code 

Partitions subarray num[first..last], returns index of pivot value 

int partition(int num[], int first, int last) 

{ 

int pivot = num[first]; 

iint t lleft ft = fi first t + 11, right i ht = llast; t 

do { 

} 

while (left < = right && num[left] < = pivot) 

++left; 

while (left < = right && num[right] > pivot) 

--right; 

Repeatedly finds 

if (left < right) 

leftmost element 

swap (num[left], num[right]); 

in left partition 

} while (left < = right); that should be in 

swap (num[first], num[right]); right and 

return right; 

rightmost that 

Move pivot into middle 

should be in left, 

David Keil Analysis of Algorithms 4. Divide and conquer swaps 1/12them 14


Quicksort recurrences 

Qsort (A) = 

A If |A| ≤ 1 

Qsort(Left-part Qsort(Left part (A (A, A[1])) + A[1] + 

Qsort (Right-part (A, A[1])) otherwise 

Left-part (A, x) = sequence of all elts < x 

λ if A = λ 

A[1]+Left-part(A[2 A[1]+Left part(A[2..|A|],x) |A|] x) if A[1] < x 

Left-part(A[2..|A|],x)+A[1] otherwise 

Right-part(A, x) mirrors Left-part(A, x) 



Complexity of Quicksort (avg. case) 

T Quick(n) = 1 if n ≤ 1 

O(n) + 2TQuick ( ⎡(n − 1) / 2⎤ ) 

otherwise 

• There are (lg n) levels of recursion because 

(n − 1) is repeatedly divided by 2 

• There are n steps at each level of recursion to 

perform partition 

• WWe can toss t out tt term 1 and df factor t 2 

• Solution: T(n) = O(n lg n) 

•This is much better than O(n2 ) 



Merge algorithm 

A and B are sorted arrays; Merge returns a 

sorted array containing all elements of A, B 

Merge (A, B) 

= 

A if B = λ 

B if A = λ 

A[1]+Merge(A[2..|A|], B) if A[1] < B[1] 

B[1]+Merge(A, B[2..|B|]) otherwise 

Spec: If f A, B are each hsorted d arrays, then h 

Merge(A, B) is a sorted array 

Problem: Write TMerge(n) recurrence 



Mergesort 

Divide-and-conquer algorithm: divide array in 

two, sort each half, merge the results 

Mergesort(A) 

If |A| ≤ 1 

return A 

else 

return Merge (Mergesort(A [1 .. |A| / /2]) 2]), 

Mergesort(A [|A| / 2 +1.. |A|])) 

Problem: Write T Mergesort(n) recurrence 



Order statistics 

• Problem: Find kth-highest value in an 

unsorted list; e.g., median 

• BBrute-force f solution: l i 

Kth-highest(A, k) = 

A[1] if k = 1 and size(A) = 1 

min-elt(A) if size(A) = k 

Kth-highest(A g ( – min-elt(A), ( ), k – 1) ) 

otherwise 



Computing order statistics 

• kth order statistic in a list is the kth element of a 

sorted version of the list 

• Median edian is sode order statistic s s cw where eekk = ⎡n / 2⎤⎤ 

• A variable-size-decrease algorithm: 

– Choose pivot as in Quicksort 

– Partition list, find pivot position p 

– If p < ki, find (k – p)th order statistic of right 

partition; recurse 

– T(n) = T(n / 2) + (n + 1) (average case) 

– T(n) ∈θ(n) [error? n vs n+1] 



Order statistic recurrence 

OS(k, A) = 

A[pivloc(A)] if k = pivloc(A) 

OS( OS(pivloc(A) ( ) – k, A[1..pivloc(A)-1] ( ) 

if k < pivloc(A) 

OS(k – pivloc(A), A[pivloc(A)+1, |A|) 

otherwise 

Pivloc is computed by using the 

Partition from Quicksort 



2. Tree algorithms 

• Tree traversals: Recursively visit 

subtrees until subtrees are leaves 

• Binary search tree search: Makes 

use of BST ordering property and 

average height of binary tree 

• Heap algorithms: Make use of 

height of complete binary tree 



Inorder traversal 

Inorder-traverse (node, op) 

1. If node has left child 

Inorder-traverse (Left(node), op) 

2. Apply operation op to node 

3. If node has right child 

Inorder-traverse (Right(node), op) 


(Left, parent, right) 

• Order for tree above: 1, 3, 5, 6, 7 

• Application: Display BST in ascending order 


Preorder traversal 

A node is 

operated on as 

soon as it is 

first visited. visited 

Preorder-traverse (node, op) 

1. Apply operation op to node 


Preorder-traverse (Left(node), op) 


Preorder-traverse (Right(node), op) 


• Application: Copy tree 



Postorder traversal 

A node is 

operated on 

after its 

descendants 

Postorder-traverse (node, op) 


Postorder-traverse (Left(node), op) 


Postorder-traverse (Right(node), op) 

3A 3. Apply l operation ti op tto node d 


• Application: Deallocate tree 



BST search of a subtree 

Pointer to subtree root 

BST-search (root, key) 

If root is null, return false 

If root’s data matches key 

return true 

otherwise 

if root’s data > key 

return BST-search (left (root), key)) 

otherwise th i 

return BST-search (right (root), key)) 

Search key 

• Initial value of root is root of entire tree 



Complexity of BST-search 

TSrch(n) = Θ(1) if n < 2 

Θ (1) + T TSSrch(n h(n /2) / 2) otherwise 

= Θ(lg n) 


• What are the assumptions of the above? 

• Does it apply to best, worst, average cases? 

• Compare with binary search 

• How does BST-insert compare? 


A heap is a kind of 

complete binary tree 

• Solves priority queue problem 

• A complete l bi binary tree has h all lll levels l full f llexcept 

possibly the bottom one, which is filled from the 

left 

• Which are complete binary trees? 




Array implementation of 

complete binary tree Java 

• Root is A[1] A[0] 

• A[i]’s left child is A[ 2i ]; A[2*i + 1] 

• Its right child is A[2i + 1] A[2*i + 2] 

• Parent of A[ i ] is A[⎣i / 2⎦] A[(i-1)/2] 

• If n is size, , height g is ⎡log ⎡ g2n⎤ 2 ⎤ 


Height of a complete binary 

tree with n nodes is O(log 2n) 

• …because size n approximately 

pp y 

doubles with each level added 

•Or, 2height−1 ≤ n ≤ 2height −1 



A maximum heap 

• Heap property: for all i up to array size, 

A[i] ≤ A[i / 2] 



The Heapify operation 

• Heapify(A,i) applies to one node, with 

subscript i, of a complete binary tree stored 

iin array A 

• It assumes that subtrees with roots Leftchild(i) 

and Right-child(i) already have the 

heap property 

• Node A[i] may meet or violate heap property 

• Heapify(A,i) causes the subtree with root 

subscript i to have the heap property 



Intuition for Heapify 


• The algorithm drags the value at a selected 

node down to its proper level where heap 

propert property applies to it 

• It does this by recursively exchanging the 

value in the root node with the higherpriority 

child of its two children 

• CComplexity: l it DDepth th of f complete l t binary bi tree t 

with n nodes, i.e., Θ(lg n) 


Heapify(A,i) for minimum heap 

L ← Left( i ) 

R ← Right( i ) 

If L = null and R = null then return 

if L ≤ |A| and A[L] <


To convert an array to a heap 

Overview: Heapify at every non-leaf 

node, starting at the bottom 

Build-heap(A) 

for i ← ⎣|A| / 2⎦ down to 1 

Heapify(A,i ) 

Because leaves don’t 

need to be heapified 

[SHOW WHY] 

Use version for min 

or max heap 

Analysis: Θ(n lg n) 



Extract value from heap 

Extract-min (A) 

If |A| < 1 

throw exception 

min ← A[1] 

A[1] ← A[|A|] 

|A| ← |A| − 1 

Heapify(A,1) 

Return min 

• Returns minimum 

value from a min-heap 

• Deletes that value 

from heap 

• Analysis: Θ(lg n) 



Insertion into a min-heap 

Heap-insert (A, key) 

Heap-size(A) ← Heap-size(A) + 1 

i ← Heap-size(A) 

while i > 1 and A[Parent( i )] > key 

A[i ] ← A[Parent( i )] 

i ← Parent( i ) 

A[i ] ← key 


Analysis: Θ(lg n) 

because Parent[i] halves i 


To sort an array using a max-heap 

Heapsort(A) 

Build-heap(A) 

using 

for i ← length(A) g ( ) downto 2 

exchange A[1] with A[ i ] 

Heap-size(A) ← Heap-size(A) − 1 

Heapify(A,1) 

Analysis: Θ(n lg n) 

• In plain language… 

Repeatedly extract the highest value 

from heap and put it at front of a 

growing sorted array just after the heap 

Replace 

extracat 



Terminology about heaps 

Johnsonbaugh- Cormen-Leiserson- 

Schaefer Rivest 

Sift-down Heapify 

Heapify Build-heap 

Largest-delete Extract-max 

• Our reference is Cormen-Leiserson-Rivest 

• Lesson: In computer science, law of frontier 

operates for terminology 



3. Solutions to 

geometric problems 

Convex hull problem 

• The Convex hull of a set of points is a subset of 

ordered pairs (on a Cartesian plane) that define a 

polygon with no concave series of edges 

(“dents”); i.e., the posts of the shortest fence that 

would enclose all the points 

• I.e., find smallest convex polygon that contains 

all of a set of points 



Convex hull 

divide-and-conquer solution 

• Find leftmost and rightmost points in set, P 1 and 

P 2 (these are in the convex hull) 

2 ( ) 

• Define upper and lower areas as separated by the 

line segment (P 1, P 2) 

• In each area, find farthest point from segment, 

recursively find triangles defined by farthest 

points -- EXPLAIN 

• Combine upper and lower hulls 

• Average case: T(n) = θ(n lg n), worst: θ(n2 ) 


diagram 


Closest pair (of a set of points) 

• Given a set of ordered pairs (points on a Cartesian 

plane) find the two points that are closest together. 

• Algorithm: g 

1. Divide set into two vertical areas 

2. Find closest pair in each 

3. Choose closest of these two 

4. See if any pairs crossing boundary are closer 

• T(n) = 2T(n / 2) + M(n), where M(n) is time to 

merge the solutions to subproblems 

• T(n) ∈θ(n lg n) 

• Brute force approach yields θ(n2 ) 



4. Graph algorithms 

• Depth-first search: Label each vertex of 

graph with a number in order visited, using 

the strategy of exhausting one path before 

backtracking to start at a new edge 

• Breadth-first search: Label each vertex, 

using the strategy of visiting each vertex 

adjacent to the current one at each stage 

• Topological sort of a dag (2 versions): see 

below 



Depth-first search of graph 

GDFS(G) G = (V, E) 

> Pre: there is a vertex v in V s.t. Mark[v] = 0 

count ← 0 

for each vertex v ∈ V do 

if Mark(v) = 0 

vdfs(v, count) 

return Mark 

> Post: Vertices are marked in order of 

> some DFS traversal 

Running time: Θ(n) 




Depth-first search from vertex 

vdfs(v, V ) // recursive 

count ← count + 1 

Mark[v] ← count 

For each vertex w ∈ V adjacent to v do 

if Mark[w] = 0 

vdfs(w, V) 

> Post: v’s mark is set 

Challenge: show termination 

[See URL found by B. Grozier] 


Breadth-first search 

• Checks all vertices 1 edge from origin, then 

2 edges, then 3, etc., to find path to 

destination 

• If G = 〈{a..i}, (a,b), (a,c), (a,d), (c,d), (b,e), 

(b, f ), (c, g), (g, h), (g, i), (h, i) 〉, and 

– BFS input is G, a, f, then 

– Order Odeof o search seac is s(a,b), (a,b), (a,c), (a,d), (b,e), 

(b, f ) 

[pic] 



Breadth-first search 

BFS(G) 

count ← 0 

G = (V, E) Iterative 

for each vertex v ∈ V do 

if Mark(v) = 0 • Proceeds concentrically 

bfs(v) 

bfs(v) 


Mark (v) ← count 

Queue ← ( v ) 

While not empty(queue) do 


• UUses queue rather h than h 

DFS’s stack 

• See example (pic) 

For each vertex w ∈ V adjacent to front(queue) do 

if Mark(w) = 0 


append w to queue 

Remove front of queue 


Topological sort of dag 

• dag: directed acyclic graph 

• Example: a listing of courses in a prerequisite 

relation so that no course is taken before its 

prereqs 

• Algorithm 1: Perform DFS, arrange vertices 

in reverse order of becoming dead-ends 

• Algorithm 2: Find source (vertex lacking inedges), 

delete it and its adjacent vertices 

recursively. Order of deletion is topological 

ordering 



5. Decrease by constant factor 

• Looking for solutions where 

T(n) = T(n / b) + f (n) 

• If T( T(n) ) = T( T(n/2) /2) + f ( (n), )th then depth d thof frecursion i 

is log2n • Examples: 

– Binary search, where f(n) = 1, 

hence solution is O(lg (g n) ) 

– Quicksort, where f (n) = n, 

hence solution is O(n lg n) 



Master Theorem (“Main 

Recurrence Theorem”) 

•Let T(n) = aT(⎣n/b⎦) + f (n) 

• Let f (n) ∈ θ(nd Let f (n) ∈ θ(n ), d ≥ 0 

• Then T(n) ∈ 

– θ(nd ), if a < bd – θ(nd lg n), if a = bd – θ(n logba), if a > bd • This theorem simplifies solving some 

recurrence equations for divide-and-conquer 

algorithms 

• Example: Binary search 



Applying the Master Theorem 

• Let T(n) = aT(n/b) + f (n), f (n) ∈θ(nd ) 

• Then by Master Thm., T(n) ∈θ(nd lg n), if a = bd • EExamples: l 

– (Binary search) T(n) = T(n/2) + O(1), 

so a = 1, b = 2, d = 0, hence a = bd , f(n) = 1, 

hence T(n) ∈θ(lg n) 

– (Quicksort) T(n) = 2T(n/2) + O(n), 

so a = 2, b = 2, d = 1, hence a = bd , , , 

• Master Theorem does not apply to linear search, 

quadratic sorts, or exponential-time algorithms 



Servant Conjecture (Keil) 

•Let T(n) = T(n − k) + f (n), with k constant, 

f (n) ∈θ(n d ) 

• Th Then T( T(n) ) ∈ θ( θ(n f ( (n)) )) 

•Examples: 

– Linear search is θ(n) 

– Bubble, Selection, Insertion sorts are 

θ(n2 θ(n ) 2 ) 

• The Servant Conjecture is useful, and is 

probably easily proven – try it! 



binary search 

binary-tree traversal 

breadth-first search 

BST search 

dag 

decrease by constant 

factor 

depth-first search 

di directed d acyclic li graphh 

divide and conquer 

Concepts 


extract-min 

heap 

heapify pfy 

heapsort 

inorder traversal 

logarithmic time O(lg n) 

Master Theorem 

mergesort 

order d statistic i i 

Quicksort 

topological sort 

tree traversal 


References 

Cormen, Leiserson, Rivest. Algorithms. MIT 

Press, 1997. 

A. Levitin. The Design and Analysis of 

Algorithms. Addison Wesley, 2003. 

Chapters 4-5. 

R. Johnsonbaugh and M. Schaefer. 

Al Algorithms. ith PPearson PPrentice ti Hall, H ll 2004. 2004 

Chs. 3-6. 

Notation ideas by A. Bilodeau, 2006.

4. Divide and conquer - Framingham State University

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?