here - Stefan-Marr.de

8.6. Absolute Performance
To conclude, direct performance gains depend to a large extent on the application
and the concurrency model implemented on top of the OMOP. Thus,
the modest overall performance gain might not justify the inherent overhead
for other applications. However, the original motivation was to improve the
debugging experience during the VM implementation by avoiding unnecessary
reification. For this purpose, this optimization still provides the desired
benefits so that debugging can focus on the relevant reified operations.
8.6. Absolute Performance
The final question for this performance evaluation is about which of the implementation
strategies yields better performance in practice. Sec. 8.4 showed
that direct VM-support brings the OMOP-based implementation of LRSTM
on par with its ad hoc implementation on top of the RoarVM. However,
Sec. 8.3 showed that the CogVM is about 11.0x faster than the RoarVM (opt),
but the AST-transformation-based implementation (AST-OMOP) brings a performance
penalty of 254.4x for enforced execution, which translates to a slowdown
of 2.8x compared to the ad hoc implementations. This experiment compares
the absolute performance of the benchmarks running on top of the
CogVM and the AST-OMOP implementation with their performance on top
of the RoarVM+OMOP (opt).
Fig. 8.11 shows the results of the AmbientTalkST and LRSTM being executed
with identical parameters on CogVM with AST-OMOP and on the
RoarVM+OMOP (opt). The benchmark results have been normalized to the
mean of the corresponding result for the RoarVM+OMOP (opt). To improve
the plot’s readability, only the results for the CogVM are depicted.
They show that six of the microbenchmarks and one of the kernel benchmarks
achieve higher absolute performance on the RoarVM+OMOP (opt).
However, the majority maintains its higher absolute performance on top of
the CogVM. For the kernel benchmarks, the benchmarks require on average a
5.7x higher runtime on the RoarVM+OMOP (opt). The AmbientTalkST NBody
benchmark is about 21% slower, while the AmbientTalkST Fannkuch benchmark
is about 14.3x faster.
To conclude, the AST-transformation-based implementation is currently the
faster implementation because it can benefit from the overall higher performance
of the CogVM. The results for RoarVM+OMOP (opt) are not optimal
but an integration into the CogVM or other VMs with JIT compilation might
result in better overall performance. This assumption is further supported by
227