here - Stefan-Marr.de

8. Evaluation: Performance
all OMOP operations and does not benefit from an optimization similar to
the customization check that is done by the RoarVM+OMOP implementation
(cf. Sec. 7.2.1 and Sec. 8.5.3). In this case, the ad hoc implementations are more
precisely tailored to the needs of the actor or STM system and do not show
the same overhead.
Results for Kernel Benchmarks The kernel benchmarks depicted in Fig. 8.5
show a very similar result, even though the results are less extreme than for
the microbenchmarks. The VM-based implementation comes very close to
the ideal performance of the ad hoc implementations. The average overhead
is about 28% on the given benchmarks. However, the overhead ranges from as
low as 2% up to a slowdown of 2.6x. The overhead greatly depends on the requirements
of AmbientTalkST and LRSTM with respect to which intercession
handlers of the OMOP have to be customized. Considering only the LRSTM
benchmarks, they have an average slowdown of 11% (min 2%, max 17%),
which is lower than the inherent overhead of 17% caused by the VM changes
that add the OMOP (cf. Sec. 8.5.2). Thus, the OMOP provides a suitable unifying
substrate that can facilitate efficient implementation of abstractions for
concurrent programming.
However, the AmbientTalkST implementation is about 48% (min 20%, max
2.6x) slower on the RoarVM+OMOP. Sec. 5.5 and Sec. 9.5.3 discuss potential
optimizations that could reduce this overhead for all concurrency concepts
that are similar to actors, Clojure agents, and CSP.
While the RoarVM+OMOP exhibits good performance characteristics compared
to the ad hoc implementations, the AST-OMOP performs significantly
worse. While the Fannkuch (AT) benchmark shows 5% speedup, the average
runtime is 2.8x higher than the runtime of corresponding ad hoc implementations.
For the NBody (AT) benchmark it is even 17.4x higher.
Conclusion The experiments show that OMOP-based implementations can
reach performance that is on par with ad hoc implementations. Thus, the
OMOP is a viable platform for the implementation of concurrent programming
concepts. The results indicate that a VM-based implementation can lead
to the ideal case of performance neutral behavior while providing a common
abstraction. However, there is currently a sweet spot for abstractions similar
to an STM that customize state accesses.
Furthermore, results show that an AST-transformation-based implementation
can show a significant performance overhead. However, this may be ac-
216