here - Stefan-Marr.de

6. Evaluation: The OMOP as a Unifying Substrate
Implementation size is often assumed to be an indication for implementation
complexity, however, this assessment has an inherently social and subjective
component. Therefore, the evaluation here considers only directly measurable
aspects. It uses implementation size in terms of lines of code (LOC)
as a surrogate measure for implementation complexity. As pointed out by a
number of studies, size seems to correlate with other metrics to such a degree
that it is not clear whether there is value in these additional metrics
to assess complexity. Jay et al. [2009] found a strong correlation between cyclomatic
complexity [McCabe, 1976] and lines of code. For their experiments,
they go so far as to conclude that cyclomatic complexity has no explanatory
power on its own. van der Meulen and Revilla [2007] found that Halstead Volume
[Halstead, 1977] and cyclomatic complexity correlate directly with LOC
on small programs. Emam et al. [2001] studied object oriented metrics and
their relation to class size, i. e., LOC. They find that previous evaluations of
metrics did not account for class size and that the effects predicted by these
metrics can be explained based on size alone. They conclude that the value of
these metrics needs to be reevaluated.
Since these indications cast doubt on the value of metrics other than size
for assessing complexity, this evaluation uses only size-based metrics as a
surrogate for measuring complexity to compare the ad hoc and OMOP-based
implementations.
The second aspect of applicability is the ability to vary language guarantees.
Sec. 6.5.1 discusses how the OMOP can be used to vary the semantics of
concurrent programming concepts. The goal is to argue that it enables a wide
range of variations covering a significant part of the design space spanned by
the supported concurrent programming concepts. Based on the agents case
study, the section argues that the additional semantic guarantees provided
over Clojure agents are the kind of variations that are desirable and a good
indication for the variability of the OMOP. The range of supported concepts
is an other indication that supports the conclusion that the OMOP provides
sufficient flexibility for the definition of language semantics for concurrent
programming.
Relevance of supported Concepts To evaluate the relevance, Sec. 6.5.1 argues
that the supported concepts are used today. It gives for each of the
supported concepts an example of either recent research in the corresponding
area, or a programming language that is respected and used in industry.
Based on these indications for actual use, it argues that each of the supported
140