here - Stefan-Marr.de

8. Evaluation: Performance
The goal of the applied optimizations is to ensure that the performance overhead
introduced by the OMOP is not hidden by the interpretation overhead
and other performance reducing factors. The main assumption here is that
the OMOP introduces a constant amount of time as overhead for each of the
adapted operations. Therefore, a reduction of the overall execution time more
clearly exposes the introduced constant overhead per operation.
The RoarVM (opt) configuration does not support parallel execution, drops
the use of the object table, comes without garbage collection, and uses direct
access to important globals such as the interpreter object, and the memory
system object. While these settings might sound like a strong restriction and
an artificial setup, they are close to the settings for a experimental version of
the RoarVM that supports parallel execution. One insight during the work
on the RoarVM was that access to thread-local variables carries a significant
performance penalty compared to direct access to global variables in a sequential
version. Hence, we implemented a variant of the RoarVM that uses
processes instead of threads to enable the utilization of multiple cores. This
variant does not pay the penalty of thread-local variables and can benefit from
compiler optimizations for statically known global variables. Thus, using direct
access to global variables is one optimization applied to the RoarVM (opt).
By avoiding the overhead of the object table, it gains additional performance.
While the current GC implementation is not adapted to work without an object
table, supporting this setting is relatively straightforward and common
in other VMs. Therefore, the configuration of RoarVM (opt) is not an artificial
one, but is based on reasonable assumptions and can be made to support
parallel execution. Furthermore, by improving overall performance any additional
constant overhead introduced by the OMOP can be determined more
clearly.
8.1.4. Generalizability and Restrictions of Results
Generalizability to Application Performance Benchmarks model only certain
sets of behaviors and do not necessarily correspond to actual applications.
Thus, their generalizability is typically restricted. For these reasons Blackburn
et al. [2006] propose a suite of actual applications as benchmarks for JVMs.
Since no such benchmark suite exists for Smalltalk, the evaluation relies on
microbenchmarks and kernel benchmarks. Therefore, benchmarks can only
give an indication of the order of magnitude of performance of applications,
precluding predictions about the performance of concrete applications.
206