Quality and Reliability Methods - SAS

Chapter 10 Variability Charts 173
Attribute Gauge Charts
As an example of the calculations, consider the following table of data for three raters, each having three
replicates for one subject.
Table 10.4 Three Replicates for Raters A, B, and C
A B C
1 1 1 1
2 1 1 0
3 0 0 0
Using this table,
4 5
+
2
2
% Agreement = ----------------------
9
2
16
= ----- = 0.444
36
4 + 3+
3 10
% Agreement [rater A] = % Agreement [rater B] = --------------------- = ----- = 0.476 and
8 + 7+
6 21
4+ 3+
2
% Agreement [rater C] = ---------------------
8+ 7+
6
9
= ----- = 0.4286
21
Agreement
The Agreement Report gives agreement summarized for every rater as well as overall agreement.
The Agreement Comparisons give Kappa statistics for each Y variable compared with all other Y variables.
In other words, each rater is compared with all other raters.
The Agreement within Raters report shows the number of items that were inspected. The confidence
intervals are Score confidence intervals, as suggested by Agresti and Coull, (1998). The Number Matched is
defined as the sum of number of items inspected, where the rater agreed with him or herself on each
inspection of an individual item. The Rater Score is Number Matched divided by Number Inspected.
The simple kappa coefficient is a measure of inter-rater agreement.
where
P 0
– P e
κˆ = ----------------
1 – P e
and
P 0
=
p ii
i
P e
= p i.
p .i
i