how do adolescents define depression? - cIRcle - University of ...

Chapter III: Methodology
"something that needs to be dealt with and resolved," "more rare in Canada," "depressed," "go to
do sports," "reborn," or "unfortunate."
Intercoder Reliability
In order to establish validation of the coding scheme (Neuendorf, 2002) for participants'
constructs of COAD, a graduate student in Educational and Counselling Psychology was invited
to judge and score a random sample of 326 units of participants' COAD units of descriptions of
adolescent depression following a brief training phase by this investigator just prior to coding.
Cohen's kappa (Cohen, 1960, 1968), utilized to evaluate intercoder reliability in studies
conducting content analysis (Greener & Crick, 1999; Nauta, Hospers, Jansen, & Kok, 2000;
Neuendorf, 2002), was used to evaluate intercoder reliability. Training involved instruction about
the meaning of all the categories labeled alphabetically, using Access (2003), and coding
instructions in how to assign his or her selected category to each randomly selected unit of
COAD onto printed Access (2003) sheets of randomly selected units of adolescent depression
definitions (Appendix M). The intercoder had a list of all the categories labeled alphabetically
printed on a couple of sheets and a set of Access (2003) printed sheets containing the units that
required the alphabet letter of the category to match the unit (Appendix N). The intercoder
entered his or her choice of the different alphabet letter that was matched to each of the 18
categories against each visible, next to the randomly selected unit.
In setting up the selected sample of units in Access (2003) for the intercoder, the sample
was first randomly selected in SPSS (Version 13, 2004). The selected sample was then saved in
the Excel (2003) program and finally transferred into Access (Version 2003). The intercoder had
two columns in view, the listed raw responses in one column, and an empty column to enter his
or her choice of code or a category alphabet letter to correspond next to the raw response
(Appendix M). The random sample of 326 units of participants' definitions of adolescent
depression represented 15% of the total sample (or 15.7% of total units in this study). A sample
size between 50 and 300 is the recommended one for intercoder reliability (0.R. Holsti, personal
communication, March 16, 2006; Neuendorf, 2002; Krippendorff, 2004; Yang & Chen, 2006). In
a previous study on strategies for making friends, Wentzel and Erdley (1993) used 10% of their
total of 2,300 strategies for a sample for interrater reliability.
68