Dense Matrix Algorithms -- Chapter 8 Introduction

More documents

Recommendations

Info

6 Increasing The Granularity To Obtain A Cost Optimal Algorithm • Use few processor than n 2 , say p processors • Instead of 1 element per processor, assume the matrix is distributed with a n/√p × n/√p submatrix (block) per processor • Assume x is distributed on the last column in blocks of size n/√p in the natural way • Use the same algorithm -- the differences are: – n/√p elements of x are distributed to the diagonal processors » Takes t s + t w n/√p time (versus unit time) – n/√p elements of x are moved from the diagonal processor in a one-toall broadcast to all processors in a column » Takes (t s + t w n/√p)log√p time (versus Θ(log n)) – A matrix-vector subproduct is computed » Takes (n/√p) or n 2 /p time (versus Θ(1)) – n/√p elements of partial results for y are sum reduced to the last processor in each row » Takes (t s + t w n/√p)log√p time (versus Θ(log n)) 5/6/2003 densematrix 11 Scalability Analysis • Adding these times up, the parallel runtime is approximately n 2 /p + t s log p + t w n/√p log p • The cost (as long as p is bounded away from n) is Θ(n 2 ) -- the algorithm is cost optimal • But what is the bounding relationship – Let's develop the isoefficiency function – The overhead T o is: T o = pT p – W = t s p log p + t w n√p log p • Using this result and some algebra, we find that W = Θ(p log 2 p) – Using a "degrees of concurrency argument", W = Ω(p), a lower bound for W 5/6/2003 densematrix 12
7 Determining The Cost-Optimal Constraint on p • We now have two expressions for W coming from the requirement for cost optimality • Setting them equal, p log 2 p = O(n 2 ) • Now we need to derive an expression for p in terms of n to determine the upper bound on p in terms of n • Takes the log on both side, solve for log p and substitute back into the above equation gives: ⎛ ⎜ n p = O ⎝ log 2 2 ⎞ ⎟ n ⎠ 5/6/2003 densematrix 13 Comparison: 1-D Versus 2-D • Runtime: • 1-D: n 2 /p + t s log p + t w n • 2-D: n 2 /p + t s log p + t w n/√p log p – The 2-D partition is faster -- smaller t w term • Isoefficiency (the growth in the work to keep the efficient fixed): • 1-D: Θ(p 2 ) • 2-D: Θ(p log 2 p) – The 2-D partition is more scalable » That is, the efficiency can be maintained with fewer processors (or on a wider range of processors) 5/6/2003 densematrix 14
Page 1 and 2: 1 Dense Matrix Algorithms -- Chapte
Page 3 and 4: 3 Fewer Than n Processors • Suppo
Page 5: 5 Diagram Of The Computational And
Page 9 and 10: 9 Performance And Analysis for the
Page 11 and 12: 11 The DNS Algorithm • Partitions
Page 13: 13 The DNS Algorithm Continued 5/6/

Dense Matrix Algorithms -- Chapter 8 Introduction

Create successful ePaper yourself

Delete template?

Save as template?