Dense Matrix Algorithms -- Chapter 8 Introduction

8
Matrix-Matrix Multiplications C = AB
• Assume the best serial algorithm is: O(n 3 )
• This is not true however
– Strassen's algorithm has fewer operations but not substantially
– There are however others
• In parallel, there are three algorithms discussed:
• A simple block algorithm
– Communication contention and lots of memory -- parallel runtime Ω(n)
• Cannon's block algorithm -- reduces the memory requirement
– Allows computation/communication overlap
» Changes the parallel runtime a little unfortunately
• The DNS algorithm (Dekel, Nassimi, Sahni algorithm)
– Partitions intermediate data so that the parallel runtime is reduced to
Θ(log n) -- an upper bound lower than the lower bound for the above
two algorithms
5/6/2003 densematrix 15
The Simple Algorithm
• Assume matrices A and B of size n×n
• Assume p processors in a grid of size √p×√p
• Assume the matrices are distributed by blocks of size
n/√p×n/√p on each processor for both A and B
– Algorithm:
• Perform an all-to-all broadcast in each row of processors of the
blocks of A in the particular row
– For row i, this insures that every block of the i-th block row of A
is on every processor in the i-th row of the grid
• Perform an all-to-all broadcast in each column of processors of
the blocks of B in the particular column
– For column j, that insures that every block of the j-th block
column of B is on every processor in the j-th column of the grid
• Perform the row-block multiplication by the column -block of
the blocks on each processor -- this computes the appropriate
block of C
5/6/2003 densematrix 16