IPDE - Extranet Systems - World Health Organization

76 A. W. Loranger
Table 7. Interrater reliability and temporal stability of 1571PDE items
Interater reliability (N=141) Temporal stability (N=243)
Intraclass
R No. of Items % Cumu1.B No of Items % Cumul. %
0.90-1.00 13 8.3 0 (2) 0 (1.3)
0.80-0.89 72 45.9 54.2 2 (13) 1.3 (8.3) 1.3 (9.6)
0.70-0.79 52 33.1 87.3 14 (42) 8.9 (26.8) 10.2 (36.3)
0.6&0.69 12 7.6 94.9 62 (62) 39.5 (39.5) 49.7 (75.8)
0.50-0.59 4 2.5 97.4 53 (27) 33.8 (17.2) 83.4 (93.0)
0.40-0.49 1 0.6 98.0 19 (5) 12.1 (3.2) 95.5(96.2)
~0.40 3 1.3 7 (6) 4.5 (3.8)
Kappa' was used to measure the interrater agreement and temporal
stability of the PD diagnoses. Because of its instability in samples
with an infrequent number of cases of a disorder, the recommendation
was followed that it be calculated only when the prevalence of a disorder
is at least 5%.4 To provide more opportunities for the calculation of
kappa, it was also determined by combining definite and probable diagnoses.
The IPDE assigns a probable diagnosis when a subject meets one
criterion less than the number required for the diagnosis. An overall
weighted kappa was also determined for all PDs, including those with a
base rate of less than 5%. It was calculated by weighing each category of
PDs by the total number of cases assigned a diagnosis in that category by
either rater, regardless of whether or not the raters agreed about the diagnosis.
The numerator is the sum of the product of the diagnostic weight
and kappa for each disorder; the denominator is the sum of the weights.
The kappa values are presented in Table 9.
References
1 Ebel, R.L. Estimation of the reliability of ratings. Psychometrikn. 1951; 16:
407-23.
2 Nu~ally, J.C. Psychomtric Theov, pp. 219-20, 237-9. New York, NY:
McGraw-Hill InternationalBwkCo, 1978.
3 Fleiss. J.L. S~atisrical methods for Rates and Proportions, 2nd edn., pp. 217-20.
New York: John Wiley & Sons Inc, 1981.
4 Grove, W.M., Andreasen, N.C., McDonald-Scott, P,. Kella, M.B. & Shapiro,
R.W. Reliability sludies of psychiatric diagnosis: theny and practice. Archives
of GenemlPsychiatry, 1981; 38: 408-13.