Helge Garåsen The Trondheim Model - NSDM

of expert panels has in the past decade proved to be a well-documented method to create
consensus-based care recommendations (36-38).
In this study the two panels were asked to consider which patients could be treated without
being admitted to the general hospital based on the evaluation of referral and discharge letters.
4.1.3 Statistical analyses study I
Statistical software programs used were SPSS 14.0 and 15.0 (SPSS Inc., Chicago, IL, USA)
and Excel 2003 for Windows. Statistical significance was set at p=0.05.
There were no sample size estimations as this study was meant as a pilot to study II. The
included number of letters was chosen from a strict capacity approach as the panellists had
limited time available.
The degree of agreement was calculated by using kappa (κ) (123). To investigate the
structure of agreement between participants in each panel and between panels it was decided,
during the assessments in the pilot study, to divide the assessments into three categories; low
(1-3), intermediate (4-5) and high (6-8), and the results were tabulated against each other in
contingency tables. All reproducibility assessments were performed according to these three
categories.
Data was collected on all assessments of the 25 cases assessed by both panels for interrater
and test reliability analysis. Agreement between the panels and within the panel was estimated
as observed and proportional agreement along with kappa statistics (123-126). The
summation of the 3 × 3 tables and calculations of agreement with confidence intervals (CIs)
of kappa were performed in a Microsoft Excel model. Strength of agreement (value of κ) was
defined as: very good (0.81 – 1.00), good (0.61 - 0.80), moderate (0.41 - 0.60), fair (0.21 -
0.40) and poor (below 0.20) (124). The distribution of concordance was also analysed with a
Bland-Altman diagram (124,127-128).
42