Appendix table - Harvard Medical School - Health Care Policy

Appendix table 1. Individual-level consistency of DSM-IV diagnoses of major depression based on dichotomized screening scales with
independent clinical diagnoses in key validity studies a
I. CIDI-SC
II. PHQ-9 (Spitzer et al., 1999)
III. CES-D (Radloff, 1977)
IV. HADS (Zigmond & Snaith, 1983)
AUC SN LR+ SP LR- (n) Sample b Clinical Outcome
.85 .80 8.1 .90 0.2 (206) PC SCID (First et al., 2002)
.88 .88 7.3 .88 0.1 (6,000) PC (Kroenke et al., 2001) SCID (First et al., 2002)
.86 .80 10.0 .92 0.2 (5,026) PC (Gilbody et al., 2007) SCID (First et al., 2002)
.87 .85 7.8 .89 0.2 (7,180) PC (Manea et al., 2012) SCID (First et al., 2002)
.85 .77 12.8 .94 0.2 (6,344) PC (Wittkampf et al., 2007) SCID (First et al., 2002)
.81 .89 3.3 .73 0.1 (294) PC (Schulberg et al., 1985) DIS (Robins et al., 1981)
.76 .81 2.9 .72 0.3 (425) PC (Klinkman et al., 1997) SCID (First et al., 2002)
.82 .95 3.2 .70 0.1 (179) PC (Thomas et al., 2001) DIS (Robins et al., 1981)
.94 1.0 8.3 .88 0.0 (487) PC (Beekman et al., 1997) DIS (Robins et al., 1981)
.75 .80 2.6 .69 0.3 (503) PC (Bunevicius et al., 2007) MINI (Sheehan et al., 1998)
.80 .85 3.5 .76 0.2 (501) PC (Lowe et al., 2004) SCID (First et al., 2002)
V. Other
WB-DAT .84 .79 7.2 .89 0.2 (193) MHS (Farvolden et al., 2003) SCID (First et al., 2002)
SDDS-PC .83 .90 3.9 .77 0.1 (635) PC (Broadhead et al., 1995) SCID (First et al., 2002)
WSQ .72 .85 2.1 .59 0.2 (157) INT (Donker et al., 2009) SCID (First et al., 2002)
M-3 .82 .84 4.2 .80 0.2 (647) PC (Gaynes et al., 2010) MINI (Sheehan et al., 1998)
SCL-90/PS .72 .79 2.3 .66 0.3 (1,457) MHS (Hunter et al., 2005) PDI (Othmer et al., 2000)
PDI-4 .80 .80 4.0 .80 0.2 (343) PC (Houston et al., 2011) SCID (First et al., 2002)
a SN = Sensitivity of the screening scale; LR+ = Likelihood ratio positive of the screening scale; SP = Specificity of the screening scale; LR- =
Likelihood ratio negative of the screening scale.
b
PC = Primary care screening sample; MHS = Mental health specialty screening sample; COM = Community survey; INT = Internet screening
sample
1

Appendix Table 2. Individual-level consistency of DSM-IV diagnoses of GAD based on dichotomized screening scales with gold
standard diagnoses in key validation studies a
AUC SN LR+ SP LR- (n) Sample b Clinical Outcome
I. CIDI-SC
.81 .68 11.1 .94 0.3 (206) PC SCID (First et al., 2002)
II. GAD-7 (Spitzer et al., 2006)
.85 .89 4.9 .82 0.1 (2,740) PC (Kroenke et al., 2007) SCID (First et al., 2002)
.74 .83 2.4 .65 0.3 (157) INT (Donker et al., 2011) CIDI (Robins et al., 1988)
.79 .83 3.3 .75 0.2 (343) PC (Houston et al., 2011) SCID (First et al., 2002)
III. Other
WB-DAT .78 .63 10.5 .94 0.4 (193) MHS (Farvolden et al., 2003) SCID (First et al., 2002)
SDDS-PC .83 .90 3.9 .77 0.1 (635) PC (Broadhead et al., 1995) SCID (First et al., 2002)
WSQ .78 .93 1.7 .45 0.2 (157) INT (Donker et al., 2009) CIDI (Robins et al., 1988)
a SN = Sensitivity of the screening scale; LR+ = likelihood ratio positive of the screening scale; SP = specificity of the
screening scale; LR- = likelihood ratio negative of the screening scale.
b
PC = Primary care screening sample; MHS = Mental health specialty screening sample; COM = Community survey; INT =
Internet screening sample
2

Appendix Table 3. Individual-level consistency of DSM-IV diagnoses of panic disorder based on dichotomized screening scales with
independent clinical diagnoses in key validity studies a
I. CIDI-SC
II. GAD-7 (Spitzer et al., 2006)
III. HADS
AUC SN LR+ SP LR- (n) Sample b Clinical Outcome
.85 .76 12.3 .94 0.2 (206) PC SCID (First et al., 2002)
.86 .76 19.0 .96 0.2 (499) PC (Lowe et al., 2003) SCID (First et al., 2002)
.77 .74 3.9 .81 0.3 (965) PC (Kroenke et al., 2007) SCID (First et al., 2002)
.82 .80 4.7 .83 0.2 (499) PC (Lowe et al., 2003) SCID (First et al., 2002)
.88 1.0 4.3 .77 0.0 (503) PC (Bunevicius et al., 2007) MINI (Sheehan et al., 1998)
IV. Other
ANS c .69 .79 1.9 .59 0.4 (469) PC (Stein et al., 1999) CIDI (Robins et al., 1988)
WB-DAT .84 .75 12.5 .94 0.3 (193) MHS (Farvolden et al., 2003) SCID (First et al., 2002)
SDDS-PC .79 .78 3.9 .80 0.3 (635) PC (Broadhead et al., 1995) SCID (First et al., 2002)
WSQ .76 .90 1.6 .44 0.2 (157) INT (Donker et al., 2009) CIDI (Robins et al., 1988)
SCL-90/DS .76 .79 2.9 .73 0.3 (1,457) MHS (Hunter et al., 2005) PDI (Othmer et al., 2000)
a SN = Sensitivity of the screening scale; LR+ = Likelihood ratio positive of the screening scale; SP = Specificity of the screening scale; LR- =
Likelihood ratio negative of the screening scale.
b PC = Primary care screening sample; MHS = Mental health specialty screening sample; COM = Community survey; INT = Internet screening
sample
c Results from three separate samples were presented in this report. SN and SP were averaged over these samples to create the entries in this
row. The sample size of 469 is the combined size of all three samples. AUC, LR+, and LR- were calculated from the aggregate estimates of
SN and SP.
3

Appendix Table 4. Individual-level consistency of DSM-IV diagnoses of BPD based on dichotomized screening scales with independent
clinical diagnoses in key validity studies a
I. CIDI-SC
II. MDQ (Hirschfeld et al., 2000)
III. MSQ (Parker et al., 2006)
AUC SN LR+ SP LR- (n) Sample b Clinical Outcome
.86 c .74 61.7 .99 0.3 (206) PC SCID (First et al., 2002)
.96 c .95 31.6 .97 0.1 (206) PC SCID (First et al., 2002)
.62 .25 25.0 .99 0.8 (1,066) COM (Dodd et al., 2009) SCID (First et al., 2002)
.62 .28 9.3 .97 0.7 (695) COM (Hirschfeld et al., 2003) SCID (First et al., 2002)
.75 .58 7.6 .93 0.4 (180) PC (Hirschfeld et al., 2005;
SCID (First et al., 2002)
Zimmerman et al., 2009)
.81 .76 5.4 .86 0.3 (127) MHS (Twiss et al., 2008;
CLDX d
Zimmerman et al., 2009)
.66 .63 2.1 .70 0.5 (1,040) MHS (Parker et al., 2012;
CLDX d
Zimmerman et al., 2009)
.75 .79 2.7 .71 0.3 (179) MHS (Parker et al., 2008;
CLDX d
Zimmerman et al., 2009)
.81 .73 7.3 .90 0.3 (198) MHS (Hirschfeld et al., 2000; SCID (First et al., 2002)
Zimmerman et al., 2009)
.82 .74 7.4 .90 0.3 (96) MHS (Weber Rouget et al., 2005; SCID (First et al., 2002)
Zimmerman et al., 2009)
.62 .58 1.8 .67 0.6 (73) MHS (Miller et al., 2004;
SCID (First et al., 2002)
Zimmerman et al., 2009)
.52 .11 1.6 .93 1.0 (103) MHS (Miller et al., 2011;
SCID (First et al., 2002)
Zimmerman et al., 2009)
.74 .63 4.2 .85 0.4 (480) MHS (Miller et al., 2011;
Zimmerman et al., 2009)
SCID (First et al., 2002)
.73 .86 2.1 .60 0.2 (247) MHS (Parker et al., 2008;
CLDX d
Zimmerman et al., 2009)
.89 .81 40.5 .98 0.2 (157) MHS (Parker et al., 2006;
CLDX d
Zimmerman et al., 2009)
.81 .85 3.9 .78 0.2 (1,040) MHS (Parker et al., 2012;
CLDX d
Zimmerman et al., 2009)
IV. Other
M-3 .79 .88 2.9 .70 0.2 (647) PC (Gaynes et al., 2010) MINI (Sheehan et al., 1998)
SCL-90/DS .66 .75 1.8 .58 0.4 (1,457) PC (Hunter et al., 2005) PDI e (Othmer et al., 2000)
HPS-6 .81 .89 3.3 .73 0.2 (127) MHS (Miller et al., 2011;
Zimmerman et al., 2009)
SCID (First et al., 2002)
4

a SN = Sensitivity of the screening scale; LR+ = Likelihood ratio positive of the screening scale; SP = Specificity of the screening scale; LR- =
Likelihood ratio negative of the screening scale.
b PC = Primary care screening sample; MHS = Mental health specialty screening sample; COM = Community survey; INT = Internet screening
sample
c Entries in the first row are for the threshold designed to minimize the difference between CIDI-SC and SCID prevalence estimates. Entries in the
second row are for the lower threshold designed to increase SN relative to its value at the first threshold.
d CLDX = The outcome was a clinical diagnosis of BPD made by the treating clinician
e Unlike the other clinical assessments for BPD, which assessed BP-I and BP-II and in some cases BP NOS, the PDI was used only to assess BP-
I. It is noteworthy that a number of the other studies (Twiss et al., 2008; Weber Rouget et al., 2005) documented that screening scale
performance is considerably better in detecting BP-I than BP-II or BPD NOS. The AUC reported here for the SCL-90/DS should consequently be
considered an upper-bound estimate.
5

Appendix References
Beekman AT, Deeg DJ, Van Limbeek J, Braam AW, De Vries MZ, Van Tilburg W (1997).
Criterion validity of the Center for Epidemiologic Studies Depression scale (CES-D): results
from a community-based sample of older subjects in The Netherlands. Psychol Med 27, 231-235.
Broadhead WE, Leon AC, Weissman MM, Barrett JE, Blacklow RS, Gilbert TT, Keller
MB, Olfson M, Higgins ES (1995). Development and validation of the SDDS-PC screen for
multiple mental disorders in primary care. Arch Fam Med 4, 211-219.
Bunevicius A, Peceliuniene J, Mickuviene N, Valius L, Bunevicius R (2007). Screening for
depression and anxiety disorders in primary care patients. Depress Anxiety 24, 455-460.
Dodd S, Williams LJ, Jacka FN, Pasco JA, Bjerkeset O, Berk M (2009). Reliability of the
Mood Disorder Questionnaire: comparison with the Structured Clinical Interview for the DSM-
IV-TR in a population sample. Aust N Z J Psychiatry 43, 526-530.
Donker T, van Straten A, Marks I, Cuijpers P (2009). A brief Web-based screening
questionnaire for common mental disorders: development and validation. J Med Internet Res 11,
e19.
Donker T, van Straten A, Marks I, Cuijpers P (2011). Quick and easy self-rating of
Generalized Anxiety Disorder: validity of the Dutch web-based GAD-7, GAD-2 and GAD-SI.
Psychiatry Res 188, 58-64.
Farvolden P, McBride C, Bagby RM, Ravitz P (2003). A Web-based screening instrument for
depression and anxiety disorders in primary care. J Med Internet Res 5, e23.
First MB, Spitzer RL, Gibbon M, Williams JBW (2002). Structured Clinical Interview for
DSM-IV Axis I Disorders, Research Version, Non-patient Edition (SCID-I/NP). Biometrics
Research, New York State Psychiatric Institute: New York, NY.
6

Gaynes BN, DeVeaugh-Geiss J, Weir S, Gu H, MacPherson C, Schulberg HC, Culpepper
L, Rubinow DR (2010). Feasibility and diagnostic validity of the M-3 checklist: a brief, selfrated
screen for depressive, bipolar, anxiety, and post-traumatic stress disorders in primary care.
Ann Fam Med 8, 160-169.
Gilbody S, Richards D, Brealey S, Hewitt C (2007). Screening for depression in medical
settings with the Patient Health Questionnaire (PHQ): a diagnostic meta-analysis. J Gen Intern
Med 22, 1596-1602.
Hirschfeld RM, Cass AR, Holt DC, Carlson CA (2005). Screening for bipolar disorder in
patients treated for depression in a family medicine clinic. J Am Board Fam Pract 18, 233-239.
Hirschfeld RM, Holzer C, Calabrese JR, Weissman M, Reed M, Davies M, Frye MA, Keck
P, McElroy S, Lewis L, Tierce J, Wagner KD, Hazard E (2003). Validity of the mood
disorder questionnaire: a general population study. Am J Psychiatry 160, 178-180.
Hirschfeld RM, Williams JB, Spitzer RL, Calabrese JR, Flynn L, Keck PE, Jr., Lewis L,
McElroy SL, Post RM, Rapport DJ, Russell JM, Sachs GS, Zajecka J (2000). Development
and validation of a screening instrument for bipolar spectrum disorder: the Mood Disorder
Questionnaire. Am J Psychiatry 157, 1873-1875.
Houston JP, Kroenke K, Faries DE, Doebbeling CC, Adler LA, Ahl J, Swindle R, Trzepacz
PT (2011). A provisional screening instrument for four common mental disorders in adult
primary care patients. Psychosomatics 52, 48-55.
Hunter EE, Penick EC, Powell BJ, Othmer E, Nickel EJ, Desouza C (2005). Development of
scales to screen for eight common psychiatric disorders. J Nerv Ment Dis 193, 131-135.
Klinkman MS, Coyne JC, Gallo S, Schwenk TL (1997). Can case-finding instruments be used
to improve physician detection of depression in primary care Arch Fam Med 6, 567-573.
7

Kroenke K, Spitzer RL, Williams JB (2001). The PHQ-9: validity of a brief depression
severity measure. J Gen Intern Med 16, 606-613.
Kroenke K, Spitzer RL, Williams JB, Monahan PO, Lowe B (2007). Anxiety disorders in
primary care: prevalence, impairment, comorbidity, and detection. Ann Intern Med 146, 317-325.
Lowe B, Grafe K, Zipfel S, Spitzer RL, Herrmann-Lingen C, Witte S, Herzog W (2003).
Detecting panic disorder in medical and psychosomatic outpatients: comparative validation of
the Hospital Anxiety and Depression Scale, the Patient Health Questionnaire, a screening
question, and physicians' diagnosis. J Psychosom Res 55, 515-519.
Lowe B, Spitzer RL, Grafe K, Kroenke K, Quenter A, Zipfel S, Buchholz C, Witte S,
Herzog W (2004). Comparative validity of three screening questionnaires for DSM-IV
depressive disorders and physicians' diagnoses. J Affect Disord 78, 131-140.
Manea L, Gilbody S, McMillan D (2012). Optimal cut-off score for diagnosing depression with
the Patient Health Questionnaire (PHQ-9): a meta-analysis. CMAJ 184, E191-196.
Miller CJ, Johnson SL, Kwapil TR, Carver CS (2011). Three studies on self-report scales to
detect bipolar disorder. J Affect Disord 128, 199-210.
Miller CJ, Klugman J, Berv DA, Rosenquist KJ, Ghaemi SN (2004). Sensitivity and
specificity of the Mood Disorder Questionnaire for detecting bipolar disorder. J Affect Disord 81,
167-171.
Othmer E, Penick EC, Powell BJ, Read MR, Othmer SC (2000). Psychiatric Diagnostic
Interview IV. Western Psychological Services: Los Angeles, CA.
Parker G, Fletcher K, Barrett M, Synnott H, Breakspear M, Hyett M, Hadzi-Pavlovic D
(2008). Screening for bipolar disorder: the utility and comparative properties of the MSS and
MDQ measures. J Affect Disord 109, 83-89.
8

Parker G, Graham R, Hadzi-Pavlovic D, Fletcher K, Hong M, Futeran S (2012). Further
examination of the utility and comparative properties of the MSQ and MDQ bipolar screening
measures. J Affect Disord.
Parker G, Hadzi-Pavlovic D, Tully L (2006). Distinguishing bipolar and unipolar disorders: an
isomer model. J Affect Disord 96, 67-73.
Radloff LS (1977). The CES-D Scale: a self-report depression scale for research in the general
population. Appl Psychol Meas 1, 385-401.
Robins LN, Helzer JE, Croughan J, Ratcliff KS (1981). National Institute of Mental Health
Diagnostic Interview Schedule. Its history, characteristics, and validity. Arch Gen Psychiatry 38,
381-389.
Robins LN, Wing J, Wittchen HU, Helzer JE, Babor TF, Burke J, Farmer A, Jablenski A,
Pickens R, Regier DA, Sartorius N, Towle L (1988). The Composite International Diagnostic
Interview. An epidemiologic Instrument suitable for use in conjunction with different diagnostic
systems and in different cultures. Arch Gen Psychiatry 45, 1069-1077.
Schulberg HC, Saul M, McClelland M, Ganguli M, Christy W, Frank R (1985). Assessing
depression in primary medical and psychiatric practices. Arch Gen Psychiatry 42, 1164-1170.
Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, Hergueta T,
Baker R, Dunbar GC (1998). The Mini-International Neuropsychiatric Interview (M.I.N.I.): the
development and validation of a structured diagnostic psychiatric interview for DSM-IV and
ICD-10. J Clin Psychiatry 59 Suppl 20, 22-33;quiz 34-57.
Spitzer RL, Kroenke K, Williams JB (1999). Validation and utility of a self-report version of
PRIME-MD: the PHQ primary care study. Primary Care Evaluation of Mental Disorders. Patient
Health Questionnaire. JAMA 282, 1737-1744.
9

Spitzer RL, Kroenke K, Williams JB, Lowe B (2006). A brief measure for assessing
generalized anxiety disorder: the GAD-7. Arch Intern Med 166, 1092-1097.
Stein MB, Roy-Byrne PP, McQuaid JR, Laffaye C, Russo J, McCahill ME, Katon W,
Craske M, Bystritsky A, Sherbourne CD (1999). Development of a brief diagnostic screen for
panic disorder in primary care. Psychosom Med 61, 359-364.
Thomas JL, Jones GN, Scarinci IC, Mehan DJ, Brantley PJ (2001). The utility of the CES-D
as a depression screening measure among low-income women attending primary care clinics.
The Center for Epidemiologic Studies-Depression. Int J Psychiatry Med 31, 25-40.
Twiss J, Jones S, Anderson I (2008). Validation of the Mood Disorder Questionnaire for
screening for bipolar disorder in a UK sample. J Affect Disord 110, 180-184.
Weber Rouget B, Gervasoni N, Dubuis V, Gex-Fabry M, Bondolfi G, Aubry JM (2005).
Screening for bipolar disorders using a French version of the Mood Disorder Questionnaire
(MDQ). J Affect Disord 88, 103-108.
Wittkampf KA, Naeije L, Schene AH, Huyser J, van Weert HC (2007). Diagnostic accuracy
of the mood module of the Patient Health Questionnaire: a systematic review. Gen Hosp
Psychiatry 29, 388-395.
Zigmond AS, Snaith RP (1983). The hospital anxiety and depression scale. Acta Psychiatr
Scand 67, 361-370.
Zimmerman M, Galione JN, Ruggero CJ, Chelminski I, McGlinchey JB, Dalrymple K,
Young D (2009). Performance of the mood disorders questionnaire in a psychiatric outpatient
setting. Bipolar Disord 11, 759-765.
10