Synergy User Manual and Tutorial. - THE CORE MEMORY

Synergy User Manual and Tutorial
Time = s G +N p p G
s G
N p p G
Single Processor
s G
p G
N Processors
Time = s G + p G = 1
Under Gustafson’s proposal, increasing the number of processors has little affect on cost
or efficiency and an almost linear speedup, as shown in the graphs above. The problem
with this method of evaluating computational speedup is that the serial and parallel
programs perform different numbers of operations on the primary task because the task
for the parallel implementation is N p times larger than that of the serial. If the
parallelized operation were matrix multiplication on n 2 matrices for n s = 10, there would
be 10 3 = 1000 multiplication and 1000 addition operations in the serial program. If you
scale up the problem for N p = 4 processors the multiplication operations must increase to
4000 and the matrix n p size must increase to:
3
4000 =
3
1000 ×
3
4 = 10×
1.5874 ≈ 16
Because matrix multiplication is O(n 3 ) complexity, increasing the size of the matrix, even
minimally, creates a much bigger job. An observation by Yuan Shi was proposed in [ lxiv ],
where an equivalence between Amdahl’s Law and Gustafson’s Law is explained. The
relationship is based on the adjustment to the serial fraction in Amdahl’s Law, call it F sA ,
and the unadjusted serial fraction used in Gustafson’s Law, call it F sG , such that:
F
sA
1
=
(1 − FsG
) × N
1+
F
sG
p
As an example, consider a task that has serial fraction F sG = 0.05 with 1024 processors.
Amdahl’s Law would predict speedup S to be:
106