Synergy User Manual and Tutorial. - THE CORE MEMORY

Synergy User Manual and Tutorial
The graphs have values N p of 4, 16, 64, 256 and 1024. Notice that as the value N p
increases, the area under the curve decreases, meaning that the non-parallizable part of
the serial program has a greater effect and the degeneration occurs faster as N p increases.
Amdahl’s intention was to show “the continued validity of the single processor approach
and of the weaknesses of the multiple processor approach”. His paper proposed
arguments to support his proposal, such as:
• “The nature of this overhead appears to be sequential so that it is unlikely to be
amenable to parallel processing techniques.”
• “A fairly obvious conclusion which can be drawn at this point is that the effort
expended on achieving high parallel performance rates is wasted unless it is
accompanied by achievements in sequential processing rates of very nearly the
same magnitude.”
Gustafson’s Law
In 1988, John L. Gustafson proposed the notion that massively parallel processing was
beneficial because Amdahl’s law implies that the parallel part of the computation and the
number of processors is independent [ lxiii ]. He proposed a formula for a scaled speedup
based on an observation that in most real world computations “the problem size scales
with the number of processors”. His proposed formula is:
S =
fraction _ serial + ( fraction _ parallel × number _ of _ processors)
( fraction _ serial + fraction _ parallel = 1)
Fs
+ (1 − Fs
) × N
p
=
F + (1 − F )
s
s
=
F + (1 − F ) × N
s
1
s
p
= F + N
s
p
− N
p
F
s
= N
p
+ ( F − N F )
s
p
s
= N
p
+ (1 − N
p
) × F
s
104