Performance and Evaluation of Lisp Systems - Dreamsongs

30
machine will perform background tasks that can be disabled, such as monitoring
the keyboard and the network. On the Xerox 1100 running INTERLISP, turning
off the display can increase Lisp execution speed by more than 30%. When using
elapsed time, one is measuring these stolen cycles as well as those going to execute
Lisp.
A sequence of timings was done on SAIL with a variety of load averages.
Each timing measures EBOX time, EBOX + MBOX (memory reference) time,
elapsed time, garbage collection time, and the load averages before and after the
benchmark. For load averages .2L10, the elapsed time, E, behaved as
E =
C(1 + K(L − 1)), L > 1;
C, L1.
That is, the load had a linear effect for the range tested. The effect of load averages
on elapsed time on other machines has not been tested.
The quality of interaction is an important consideration. In many cases the
personal machine provides a much better environment. But in others, the need
for high absolute performance means that a time-shared system is preferable.
1.4.5 Measure the Hardware Under All Conditions
In some architectures, jumps across page boundaries are slower than jumps
within page boundaries, and the performance of a benchmark can thus depend
on the alignment of inner loops within page boundaries. Further, working-set size
can can make a large performance difference, and the physical memory size can
be a more dominating factor than CPU speed on benchmarks with large working-
sets. Measuring CPU time (without memory references) in conjunction with a
knowledge of the approximate mapping from memory size to CPU + memory time
would be an ideal methodology but is difficult to do.
An often informative test is to take some reasonably small benchmarks and to
code them as efficiently as possible in an appropriate language in order to obtain
the best possible performance of the hardware on those benchmarks. This was
done on SAIL with the TAK ′ function, which was mentioned earlier. In MacLisp
the time was .68 seconds of CPU + memory time; in assembly language (heavily
optimized) it was .255 seconds, or about a factor of 2.5 better. 15
15 This factor is not necessarily expected to hold up uniformly over all benchmarks.