The psychopathology of everyday art: a quantitative Study - World ...

wider ranges: purple and brown, which had 4 or 5 point limits, are mixed colours which
are harder to define and therefore more variable to rate; raters tended to score lower. And
emotional tone, which had 4 points limit, depended on the associations of the rater
(discrepancies equal).
So, how can we define good agreement?
For this research, the standard deviation of the differences as a measure of agreement is
appropriate because the subjects themselves define their own limitations. However, this
does not facilitate comparison of the measure. Interpretation of agreement depends upon
the circumstances, it is not possible to use statistics to define acceptable agreement; it
is more important to quantify the variability of the individual data points. But it is still
necessary to define some answer to the question: how well do the raters agree? within
the larger context which gives it meaning of, how do we compare this agreement with
other measures? It is a complicated question, and there are two components to the
answer.
(1) The mean difference is an estimate of the average bias of the raters relative to the
principal rater. Here the means are mostly negligible and we can say that agreement is
excellent on average.
(2) It is essential to consider the agreement for an individual painting for which purpose
the standard deviation of the differences (S.diff; Table 16) can be used.
A rough comprehensive overall statistic can be deduced from examination of the
raw data (middle columns of tables 1-15, appendix 2). A total 480 out of 630 decisions
223