Divide and Conquer

Divide and ConquerGeneral methodDivide Split the input with n sample points into k subsets, 1 < k ≤ n, with no overlap between subsetsConquer Solve each of the k subproblems, possibly by splitting recursively• If the subproblems are small enough, solve those using a simple (non-recursive) method• Also known as the bottoming out of recursion, or arrival at a base caseCombine the result of the k subproblems to solve the original problem• Detecting a counterfeit coin– Bag of 16 coins with one of them possibly counterfeit– Counterfeit coins are lighter than genuine ones– Determine if the bag contains a counterfeit coin– Algorithm 1∗ Compare a pair of coins; if one of them is counterfeit, it will be lighter∗ Compare next pair, and so on∗ You will find the counterfeit coin in at most 8 trials– Algorithm 2: Divide and conquer∗ Divide the 16 coin instance into two instances of 8 coins each∗ Compare the weight of two sets; if same, no counterfeit coin; else divide the lighter instance into two of halfthe original size and repeat∗ Presence of counterfeit coin is determined in 1 trial∗ Identification of coin happens in four trials• Natural to express the solution as a recursive algorithm• Control abstraction for recursive divide and conquer, with problem instance Pdivide_and_conquer ( P ){if ( small ( P ) )// P is very small so that a solution is trivialreturn solution ( n );divide the problem P into k instances P1, P2, ..., Pk;return ( combine ( divide_and_conquer ( P1 ),divide_and_conquer ( P2 ),...divide_and_conquer ( Pk ) ) );}• The solution to the above problem is described by the recurrence, assuming size of P denoted by n{ g(n) n is smallT n =T n1 + T n2 + · · · + T nk + f(n)where f(n) is the time to divide n elements and to combine their solution• Recursively decompose a large problem into a set of smaller problems– Decomposition is directly reflected in analysis

Divide and Conquer 2– Run-time determined by the size and number of subproblems to be solved in addition to the time required fordecomposition– General complexity computationT n =where a and b are known constants; assume that n = b k• Example, mergesort recurrenceSolution for the mergesort recurrence: Θ(n lg n){T1 n = 1aT n/b + f(n) n > 1{ Θ(1) if n = 1T n =2T n + Θ(n) if n > 12• You can ignore extreme details like floor, ceiling, and boundary in recurrence description.• Solving recurrence relations by substitution method– Guess the form of solution and use mathematical induction to find constants– Determine upper bound on the recurrenceGuess the solution as: T n = O(n lg n)Now, prove that T n ≤ cn lg n for some c > 0Assume that the bound holds for ⌊ ⌋n2Substituting into the recurrenceT n = 2T ⌊ n2 ⌋ + n⌊ n⌋T n ≤ 2(c2Boundary condition: Let the only bound be T 1 = 1≤cn lg⌊ n⌋lg(2( n2)+ n)) + n= cn lg n − cn lg 2 + n= cn lg n − cn + n≤ cn lg n ∀c ≥ 1̸ ∃c | T 1 ≤ c1 lg 1 = 0Problem overcome by the fact that asymptotic notation requires us to proveInclude T 2 and T 3 as boundary conditions for the proofChoose c such that T 2 ≤ c2 lg 2 and T 3 ≤ c3 lg 3True for any c ≥ 2– Making a good guessT n ≤ cn lg n for n ≥ n 0T 2 = 4 T 3 = 5∗ If a recurrence is similar to a known recurrence, it is reasonable to guess a similar solutionT n = 2T ⌊ n2 ⌋ + nIf n is large, difference between T ⌊ n2 ⌋ and T ⌊ n2 ⌋+17 is relatively small∗ Prove upper and lower bounds on a recurrence and reduce the range of uncertaintyStart with a lower bound of T n = Ω(n) and an initial upper bound of T n = O(n 2 ). Gradually lower the upperbound and raise the lower bound to get asymptotically tight solution of T n = Θ(n lg n)

Divide and Conquer 3– Pitfall∗ T n = 2T ⌊ n2 ⌋ + nAssume inductively that T n ≤ cn implying that T n = O(n)⌊ n⌋T n ≤ 2c + n2≤cn + n= O(n) ⇐ wrongWe haven’t proved the exact form of inductive hypothesis T n ≤ cn– Changing variables∗ Consider the recurrenceLet m = lg n.Rename S m = T 2 mT n = 2T ⌊√ n⌋ + lg nT 2 m = 2T 2m2+ mS m = 2S m2 + mSolution for the recurrence: S m = m lg mChange back from S m to T nT n = T 2 m = S m = O(m lg m) = O(lg n lg lg n)• Solving recurrence relations by iteration method– No guessing but more algebra– Expand the recurrence and express it as summation dependent on only n and initial conditions– RecurrenceT n = 3T ⌊ n4 ⌋ + nT n = n + 3T n4⌊ n⌋= n + 3(4+ 3T ⌊ n16 ⌋ )= n + 3(⌊ n 4 ⌋ + 3(⌊ n 16 ⌋ + 3T ⌊ n 64 ⌋ ))= n + 3⌊ n 4 ⌋ + 9⌊ n 16 ⌋ + 27T ⌊ n 64 ⌋ith term is given by 3 i ⌊ n 4 i ⌋Bound n = 1 when ⌊ n 4 i ⌋ = 1 or i > log 4 nBound ⌊ n 4 i ⌋ ≤ n 4 iDecreasing geometric seriesFocus onT n ≤ n + 3 4 n + 916 n + 2764 n + · · · + 3log 4 n Θ(1)∞∑( ) i 3≤ n + Θ(n log43 ) 3 log 4 n = n log434i=0= 4n + o(n) log 4 3 < 1 ⇒ Θ(n log43 = o(n)= O(n)∗ Number of iterations to reach boundary condition

Divide and Conquer 4∗ Sum of terms arising from each level of iteration– Recursion trees∗ RecurrenceAssume n to be an exact power of 2.T n = n 2 + 2T n2( (n= n 2 + 22T n = 2T n2 + n2) 2+ 2T n4= n 2 + n22 + 4 ( (n4)) 2+ 2T n8= n 2 + n22 + n24 + 8 ( (n8= n 2 + n22 + n24 + n28 + · · ·= n 2 (1 + 1 2 + 1 4 + 1 8 + · · ·)= Θ(n 2 )The values above decrease geometrically by a constant factor.∗ RecurrenceT n = T n3 + T 2n 3 + n• The Master MethodLongest path from root to a leafn →)) 2+ 2T n16( ( ) 2 2 2n → n → · · · 13)3( 2) k3 n = 1 when k = log 3 n, k being the height of the tree2Upper bound to the solution to the recurrence – n log 3 n, or O(n log n)2– Suitable for recurrences of the formwhere a ≥ 1 and b > 1 are constants, andf(n) is an asymptotically positive function– For mergesort, a = 2, b = 2, and f(n) = Θ(n)– Master TheoremT n = aT nb+ f(n)Theorem 1 Let a ≥ 1 and b > 1 be constants, let f(n) be a function, and let T n be defined on the nonnegativeintegers by the recurrenceT n = aT n + f(n)bwhere we interpret n b to mean either ⌊ ⌋ ⌈nb or n⌉b . Then Tn can be bounded asymptotically as follows1. If f(n) = O(n log b a−ɛ ) for some constant ɛ > 0, then T n = Θ(n log b a )2. If f(n) = Θ(n log b a ), then T n = Θ(n log b a lg n)3. If f(n) = Ω(n log b a+ɛ ) for some constant ɛ > 0, and if af ( )nb ≤ cf(n) for some constant c < 1 and allsufficiently large n, then T n = Θ(f(n))∗ In all three cases, compare f(n) with n log b a∗ Solution determined by the larger of the two· Case 1: n log b a > f(n)Solution T n = Θ(n log b a ))

Divide and Conquer 5· Case 2: n log b a ≈ f(n)Multiply by a logarithmic factorSolution T n = Θ(n log b a lg n) = Θ(f(n) lg n)· Case 3: f(n) > n log b aSolution T n = Θ(f(n))∗ In case 1, f(n) must be asymptotically smaller than n log b a by a factor of n ɛ for some constant ɛ > 0∗ In case 3, f n must be polynomially larger than n log b a and satisfy the “regularity” condition that af( n b ) ≤ cf(n)– Using the master method∗ RecurrenceT n = 9T n3 + na = 9, b = 3, f(n) = nn log b a = n log 3 9 = Θ(n 2 )f(n) = O(n log 3 9−ɛ ), where ɛ = 1Apply case 1 of master theorem and conclude T n = Θ(n 2 )∗ RecurrenceT n = T 2n3 + 1a = 1, b = 3 2 , f(n) = 1n log b a = n log 3 12 = n 0 = 1f(n) = Θ(n log b a ) = Θ(1)Apply case 2 of master theorem and conclude T n = Θ(lg n)∗ RecurrenceT n = 3T n + n lg n4a = 3, b = 4, f(n) = n lg nn log b a = n log 4 3 = O(n 0.793 )f(n) = Ω(n log 4 3+ɛ ), where ɛ ≈ 0.2Apply case 3, if regularity condition holds for f(n)For large n, af( n b ) = 3 n 4 lg( n 4 ) ≤ 3 4 n lg n = cf(n) for c = 3 4Therefore, T n = Θ(n lg n)∗ RecurrenceT n = 2T n2 + n lg nRecurrence has proper form – a = 2, b = 2, f(n) = n lg n and n log b a = nf(n) = n lg n is asymptotically larger than n log b = n but not polynomially largerRatio f(n) = n lg nn log b a n= lg n is asymptotically less than n ɛ for any positive constant ɛRecurrence falls between case 2 and case 3• Technicalities in recurrences– Simplify the solution by ignoring certain details∗ Consider mergesort; if n is odd, we end up with subproblems of size ⌊n/2⌋ and ⌈n/2⌉∗ Technically correct definition of mergesort recurrence is{ Θ(1) if n = 1T n =T ⌊ n2 ⌋ + T ⌈ n2 ⌉ + Θ(n) if n > 1– We may also choose to ignore boundary conditions∗ Running time of a constant-size input is a constant∗ For sufficiently small n, we can describe the run-time asT n = Θ(1)∗ As a convenience, we may omit boundary conditions from recurrence

Divide and Conquer 6∗ We can describe the recurrence for mergesort asT n = 2T n2 + Θ(n)∗ Omitting the value of T 1 changes the exact solution for the recurrence, but not by more than a constant factor;it has no effect on the order of growthBinary search• Input characterized by an array a i , 0 ≤ i < n– Elements in a are sorted in nondecreasing order– Problem to determine whether an element k is present in a∗ If it is present, return its index j such that a j = k∗ If the element is not present, return −1• Instance of problem given byP = (n, a l , . . . , a r , k)where a l , . . . , a r are n elements in the list to be searched for k• Assume small ( P ) is true if r == l– There is only one element to be tested– In this case, solution ( P ) returns i if a i = x; otherwise it returns −1– g(1) = Θ(1)• If r − l ≥ 1, compute m = (l + r)/2 leading to three cases1. a m = k– Problem is immediately solved2. a m < k– Discard the elements whose index is smaller than m– Search for k in a[m + 1..r]3. a m > k– Discard the elements whose index is larger than m– Search for k in a[l..m − 1]• Case 2 and 3 result in only one subproblem; division takes Θ(1) time– Answer to the remaining subproblem is also the answer to the original problem; no need to combine the solutions• Problem initially invoked by bin_search ( a, 1, n, k );• Recursive Algorithm// Given an array a[l..r] of elements in nondecreasing order, 0

Divide and Conquer 7return ( r );elsereturn ( -1 );}// Reduce problem to smaller instancesm = ( r + l ) / 2;if ( k == a[m] )return ( m );}if ( k < a[m] )return ( bin_search ( a, l, m - 1, k ) );elsereturn ( bin_search ( a, m + 1, r, k ) );• Iterative version// Given an array a[0..n-1] of elements in nondecreasing order, n >= 0,// determine whether k is present, and if so, return i such that k = a[i];// else return -1Algorithm bin_search ( a, n, k ){low = 0;high = n-1;while ( low a[mid] )low = mid + 1;elsereturn ( mid );}}return ( -1 );Finding the maximum and minimum• Linear scan algorithm– Straightforward comparison – 2(n − 1) comparisons– Compare for min only if comparison for max fails– Best case: increasing order – n − 1 comparisons– Worst case: decreasing order – 2(n − 1) comparisons– Average case: 3n/2 − 1 comparisons• Divide and conquer algorithm

Divide and Conquer 8item_t a[n]maxmin ( i, j, max, min ){if ( i == j ){max = min = a[i]return}if ( i == j - 1 ){if ( a[i] < a[j] ){max = a[j]min = a[i]}else{max = a[i]min = a[j]}return}// Global array// Only one element// Only two elements// Dividemid = floor ( ( i + j ) / 2 )maxmin ( i, mid, max, min )maxmin ( mid+1, j, max1, min1 )// Conquer}if ( max < max1 ) max = max1if ( min > min1 ) min = min1return• Analyzing divide and conquer maxmin described by the recurrence⎧⎨ T n + T n + 2 n > 22 2T n = 1 n = 2⎩0 n = 1Let n = 2 k , for some k > 0T n = 2T n2 + 2= 2(2T n4 + 2) + 2= 2 2 T n2 2 + 22 + 2= 2 2 (2T n2 3 + 2) + 22 + 2= 2 3 T n2 3 + 23 + 2 2 + 2..= 2 k−1 T n∑k−1+ 2 i2 k−1i=1

Divide and Conquer 9k−1∑= 2 k−1 +i=12 i= 2 k−1 + 2(k−1)+1 − 12 − 1= 2 k−1 + 2 k − 1= n 2 + n − 1= 3n 2 − 1Merge sort• Recursive algorithm• Algorithm can be described by the following recurrence{ 2T n + cn n > 1, c is a constantT n =2a n = 1, a is a constantQuick sort• The array to be sorted is partitioned at a pivot element such that the elements at indices less than that of the pivot are lessthan the pivot while the elements with indices greater than the pivot are larger than the pivot• Quicksort eliminates the need for a subsequent merge as required by merge sort• Performance analysis– Worst case when elements are already in sorted order• Randomized quick sort– Select pivot as median of three– Select a random element as the pivot (Las Vegas algorithm)Selection• Selecting kth smallest element using partition from quicksortStrassen’s matrix multiplication• Given two n × n matrices A and B; their product C is given byC ij =– It requires n multiplications for each element in C– Total time to perform above computation is Θ(n 3 )• Divide and conquer strategy– Assume that n = 2 kn∑A ik B kjk=1

Divide and Conquer 10∗ If n ≠ 2 k , we can add enough rows and columns to make it satisfy our assumption– Each of A, B, and C is partitioned into submatrices of size n 2such that[ ] [ ] [ ]A11 A 12 B11 B 12 C11 C=12A 21 A 22 B 21 B 22 C 21 C 22– This yields eight multiplications of n 2 × n 2 matrices (T n ), and four additions of the same size of matrices (assume2cn 2 for some constant c, starting with 4 · ( n 2 )2 )– The recurrence is{ b n ≤ 2T n =8T n + cn2 n > 22– A solution of this recurrence yields T n = O(n 3 ) (no improvement)– Observation: Matrix multiplications are more expensive (O(n 3 )) than matrix additions (O(n 2 ))– Strassen’s method relies on devising a clever workaround to minimize the number of multiplications while increasingthe number of additions∗ 7 multiplications and 18 additions/subtractions– The intermediate computations are– The elements of C are given asP 1 = (A 11 + A 22 )(B 11 + B 22 )P 2 = (A 21 + A 22 )B 11P 3 = A 11 (B 12 − B 22 )P 4 = A 22 (B 21 − B 11 )P 5 = (A 11 + A 12 )B 22P 6 = (A 21 − A 11 )(B 11 + B 12 )P 7 = (A 12 − A 22 )(B 21 + B 22 )C 11 = P 1 + P 4 − P 5 + P 7C 12 = P 3 + P 5C 21 = P 2 + P 4C 22 = P 1 + P 3 − P 2 + P 6– The recurrence relation iswhere a and b are constants– Solving the recurrence{ b n ≤ 2T n =7T n + an2 n > 22T n = an 2 [1 + 7 ( 2 ( ) k−1 7 74 4) + + · · · + ] + 7 k T (1)4( lg n 7≤ cn4) 2 + 7 lg n , c is a constant= cn lg 4+lg 7−lg 4 + n lg 7= O(n lg 7 )≈ O(n 2.81 )Convex hull

Divide and Conquer 11• Structure (smallest convex polygon) used in the construction of geometric structures• Polygon: A piecewise-linear, closed curve in a plane– Curve composed of a sequence of straight line segments, or sides of polygon– Curve starts and ends at the same point– A point that is common to two sides is called a vertexDefinition 1 A polygon is defined to be convex if for any two vertices p 1 and p 2 inside the polygon, the directed linesegment from p 1 to p 2 (denoted as 〈p 1 , p 2 〉) is fully contained in the polygon.Definition 2 The convex hull of a set S of points in the plane is defined to be the smallest convex polygon containing allthe points of S.• Vertices of the convex hull of a set S of points form a [not necessarily proper] subset of S• Two variants of the convex hull problem1. Obtain the vertices (extreme points) of the convex hull2. Obtain the vertices of the convex hull in some order, such as clockwise• Obtaining extreme points of a given set S of points in a planefor each p in S{for each possible triplet of points in Sif p is not inside the triangle formed by any tripletmark p as an extreme point;}– Testing for p being inside a given triangle is performed in Θ(1) time– Number of possible triangles is Θ(n 3 )– Since there are n points, the above algorithm runs in Θ(n 4 ) time• Divide and conquer allows us to solve the convex hull problem (either form) in O(n log n) time• Geometric primitives– Let A be an n × n matrix with elements denoted by a ijDefinition 3 The ijth minor of A, denoted by A ij , is defined to be the submatrix of A obtained by deleting the ithrow and jth column.Definition 4 The determinant of A, denoted by |A|, is given by{a11 n = 1|A| =a 11 · |A 11 | − a 12 · |A 12 | + · · · + (−1) n−1 · |A 1n | n > 1– Consider the directed line segment 〈p 1 , p 2 〉 from some point p 1 = (x 1 , y 1 ) to some other point p 2 = (x 2 , y 2 ). Ifq = (x 3 , y 3 ) is another point, we say q is to the left [right] of 〈p 1 , p 2 〉 if the angle p 1 p 2 q is a left [right] turn.∗ An angle θ is said to be a left turn if θ ≤ 180; otherwise, it is considered to be a right turn∗ We can check whether q is to the left or right of 〈p 1 , p 2 〉 by evaluating the determinant of their coordinatesx 1 x 2 x 3y 1 y 2 y 3∣ 1 1 1 ∣

Divide and Conquer 12Determinant > 0. q is to the left of 〈p 1 , p 2 〉Determinant = 0. The three points are colinearDeterminant < 0. q is to the right of 〈p 1 , p 2 〉· Point p is within the triangle formed by p 1 , p 2 , and p 3 iff p is to the right of each of the three lines p 1 p 2 ,p 2 p 3 , and p 3 p 1· For any three points (x 1 , y 1 ), (x 2 , y 2 ), and (x 3 , y 3 ), the signed area formed by the corresponding triangleis given by one-half of the above determinant– Checking for point p to be inside a convex polygon Q given by vertices p 1 , p 2 , . . . , p n∗ Consider a horizontal line h from −∞ to ∞ and passing through p∗ Two possibilities1. h does not intersect any of the edges of Q· p is outside Q2. h intersects some of the edges of Q· There can be at most two points of intersection; if h intersects Q at a single point, it is considered as twopoints· Count the number of points to the left of p· If the number is even, p is outside Q; otherwise it is inside Q∗ The method takes Θ(n) time to check whether p is interior to Q• QuickHull Algorithm– Similar to quicksort– Computes the convex hull of a set X of n points in the planeconvex_hull quick_hull ( X ){// Identify p_1 and p_2 as part of the convex hullp_1 = point in X with the smallest x-coordinate valuep_2 = point in X with the largest x-coordinate valuedivide X into X_1 and X_2 such that{X_1 = set of points to the left of line segment X_2 = set of points to the right of line segment // Convex hull of X_1 is the upper hull// Convex hull of X_2 is the lower hull// Compute the upper and lower hull using the divide and conquer// algorithm called hull; union of the two hulls is the overall convex// hull}H1 = hull ( X_1, p_1, p_2 );H2 = hull ( X_2, p_2, p_1 );}return ( H1 + H2 );// Union of H1 and H2– The case of ties when more than one point has the extreme x-coordinate, can be handled as a special caseconvex_hull hull ( X, p1, p2 ){for each point p in Xcompute the area of the triangle formed by p, p1, and p2

Divide and Conquer 13p3 = p, for which the above area is maximized// In case of a tie for point with maximum area, select p3 to be the point// for which the angle p3p1p2 is maximized// Divide X into two parts X1 and X2 based on p3X1 = points in X to the left of X2 = points in X to the left of // There is no point in X1 that is to the left of both and // Remaining points are interior points and can be dropped from further// considerationH1 = hull ( X1, p1, p3 );H2 = hull ( X2, p3, p2 );}return ( H1 + H2 );– Analyzing the algorithm// Union of H1 and H2• Graham’s scan∗ Let there be m points in X 1∗ We can identify p 3 in O(m) time∗ Partitioning X 1 into two is also done in O(m) time∗ Merging the two convex hulls is done in O(1) time∗ Run time of hull on m points is given by T m∗ Size of resultant parts given by m 1 and m 2 ; m 1 + m 2 ≤ m∗ Recurrence relation is:T m = T m1 + T m2 + O(m)· Similar to quicksort∗ Worst case run-time: O(m 2 ) for m points when the partitioning is highly uneven∗ For nearly even partitioning, run-time given by O(m lg m)– Given a set of points S in a 2D plane– Identify the point p with smallest y coordinate∗ Break ties by picking the point with smallest x coordinate– Sort the points by angle subtended by points and p with the horizontal axis– Scan through sorted list starting with p, three points at a time∗ If points p 1 , p 2 , p 3 form a left turn, they all are on the hull∗ If p 1 , p 2 , p 3 form a right turn, p 2 is not on the convex hull– Example

Divide and Conquer 14p 0p 3 p 13 p 2p ❜1❜❤❤❤❤❤❤❤❤❤❤❜❜✔ ☎ ✧ ✏★ ✥✥✥✥✥✥✥✥✥✥✥✥✥✥✥✥✥✥✥✥✥✥✥✥✥✥✥✥✥✥✥✥✥✥✥✥✥☎☎☎☎☎☎☎☎☎☎☎☎☎☎☎☎☎☎☎☎ ✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔✏✏✏✏✏✏✏✏✏✏✏✏✏✏✏✏✏✏✏✏✏✏✏✏✏✏✏✏✏✏✏✧✧✧✧✧✧✧✧✧✧✧✧✧✧✧✧✧✧✧✧✧✧✧✧✧✧✧✧✧★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★p 4 p 5 p 6 p 7p 8 p 14 p 9 p 10 p 12 p 11