Cognitive and clinical predictors of success in vocational ...

Schizophrenia Research 70 (2004) 331–342
www.elsevier.com/locate/schres
Cognitive and clinical predictors of success in vocational
rehabilitation in schizophrenia
Jovier D. Evans a, *, Gary R. Bond a , Piper S. Meyer a,b , Hea Won Kim a,f ,
Paul H. Lysaker c,d , P. Joseph Gibson e , Sandra Tunis e
a Department of Psychology, Indiana University-Purdue University Indianapolis (IUPUI), LD 124, 402 N. Blackford Street, Indianapolis,
IN 46202-3275, USA
b Cecil G. Sheps Center for Health Services Research, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
c Roudebush VA Medical Center, Indianapolis, IN, USA
d Indiana University School of Medicine, USA
e Eli Lilly & Company, USA
f School of Social Work, Indiana University, USA
Received 28 May 2003; received in revised form 5 January 2004; accepted 12 January 2004
Available online 19 March 2004
Abstract
Cognitive impairments in schizophrenia appear to be associated with social problem solving, social and vocational
functioning, and psychosocial skill acquisition. The present study examined the relationship of cognitive functioning, as well as
clinical symptoms, to vocational outcomes among individuals with schizophrenia. One hundred and twelve participants with
DSM-IV schizophrenia spectrum diagnoses underwent a comprehensive neuropsychiatric evaluation after enrolling in one of
several employment programs. The neuropsychological evaluation examined verbal learning and memory, attention, speed of
information processing, and executive functioning. Clinical symptoms were evaluated with the Positive and Negative Syndrome
Scale (PANSS). Vocational outcomes were assessed 4 months after baseline assessment and included both measures of
employment outcome (e.g., earnings) and of work performance as assessed by the Work Behavior Inventory (WBI). Negative
symptoms, learning and memory performance, processing speed, and executive functioning were related to hours, weeks, and
wages earned on the job. Stepwise multiple regression analyses found that among baseline clinical and cognitive predictors,
only verbal learning and memory and cognitive disorganization symptoms were significant predictors of work behaviors 4
months later. Learning and memory were the only significant predictors of integrated employment at 4 months. These results
suggest specific aspects of cognition may be modestly predictive of vocational outcomes.
D 2004 Elsevier B.V. All rights reserved.
Keywords: Schizophrenia; Neuropsychology; Vocational outcomes; Symptoms
1. Introduction
* Corresponding author. Tel.: +1-317-274-2283; fax: +1-317-
274-6756.
E-mail address: jevans2@iupui.edu (J.D. Evans).
Cognitive impairments are gaining attention for
their possible role in the social and occupational
deficits that accompany schizophrenia. Recent re-
0920-9964/$ - see front matter D 2004 Elsevier B.V. All rights reserved.
doi:10.1016/j.schres.2004.01.011

332
J.D. Evans et al. / Schizophrenia Research 70 (2004) 331–342
views of the literature have suggested these cognitive
impairments as ‘‘rate-limiting’’ factors in success in
both social and occupational domains of outcome
(Green, 1996; Green et al., 2000). Several lines of
investigation into adaptive function in schizophrenia
have documented relationships between cognitive
functioning and performance of everyday activities
(Brekke et al., 1997; Green, 1996; Harvey et al.,
1998). Specifically, these studies have noted that executive
functioning abilities, verbal memory, and vigilance
are related to work performance, social
functioning, and performing basic self-care activities.
Several dimensions of adaptive function have been
studied in relationship to cognitive performance, including
work performance and occupational goals
(Bell and Lysaker, 1995; Lysaker et al., 1995a,b), social
skills performance (Corrigan and Green, 1993; Patterson
et al., 1997; Smith et al., 1999), and community
outcomes (Brekke et al., 1997; Green et al., 2000).
Most of this work has found that cognitive function is
associated with concurrent measures of adaptive function
such as work performance; however, relatively few
studies have attempted to examine the predictive utility
of cognitive performance on future vocational success.
Bellack et al. (1999) found that cognitive functioning
was related to competitive employment among a
group of psychiatric outpatients. Specifically, processing
speed and social judgment were related to good
vocational outcomes among a large community sample.
Other studies have also found significant relationships
among work outcomes and performance on
neuropsychological tests. Results of these studies
have determined that aspects of adaptive functioning
are multi-determined and as such are necessarily
associated with multiple domains of cognitive ability
(Bryson et al., 1998; Velligan et al., 2000a).
McGurk and Meltzer (2000) have examined the
relationship of both cognitive and clinical domains to
vocational outcomes and have found that cognitive
functioning was associated with employment status.
Specifically, full-time employment was associated
with better performance on measures of executive
functioning, working memory, and vigilance, compared
to unemployed participants. This study also
found that poor premorbid function (as assessed by
educational level), negative symptoms, and cognitive
impairments were significantly related to employment
success among people with schizophrenia. Gold et al.
(2002) found that among clients enrolled in a vocational
program who achieved employment, several
different measures of cognitive performance were
related to the total number of hours of employment.
However, cognitive measures did not predict who
would work at all. This study suggests that cognitive
measures are most predictive of employment outcomes
among those who are motivated enough to
start a job.
A recent study by McGurk et al. (2003) suggests
that the type of vocational interventions and supports
that are given to clients influences the relationship
between cognitive impairment and work outcomes.
She found that individuals with cognitive impairments
were capable of competitive employment, but that
they required more intervention by the vocational staff
than less impaired clients. Therefore, the relationship
between cognitive function and measures of employment
outcomes (e.g., attainment of a competitive job,
hours worked, and job tenure) may be moderated by
the level of support and other factors, such as employer
accommodation of work limitations (McGurk
et al., 2003). Other researchers have concluded that an
effective goal of rehabilitation programs might best be
directed toward teaching clients compensatory strategies
for their cognitive deficits rather than trying to
directly remediate these deficits (Velligan et al.,
2000b).
Another type of vocational measure concerns actual
behavior in the work place or observed in prevocational
work settings (e.g., work quality and
quantity, punctuality, dependability, cooperativeness,
etc.). These measures of work performance may be
more proximally related to cognitive status and be
measured in non-paid work situations, rather than
employment status measures (i.e., getting a job, hours
worked, job tenure, etc.) that may be more dependent
on external factors outside of the individual. For
example, the benevolence of the employer, economic
conditions, and work accommodations made at the job
may determine employment status. Bell and Bryson
(2001) found that neuropsychological performance
was significantly associated with work performance
and was moderately predictive of improvements in
work performance for participants in a 6-month vocational
rehabilitation program. Specifically, aspects
of attention, executive functioning, and working
memory were related to improvements in performance

J.D. Evans et al. / Schizophrenia Research 70 (2004) 331–342 333
at work. In addition, clinical symptoms were not
predictive of these improvements. A recent study by
these researchers found that aspects of attention were
related to initial improvements in work performance
and learning, and memory was related to later
improvements in work performance in this same 6-
month rehabilitation program (Bryson and Bell,
2003). It may be that cognitive measures are more
closely linked to actual performance on the job than to
employment outcomes. Research has also shown that
supervisor ratings of work performance are related to
employment outcomes (Bond and Friedmeyer, 1987;
Bryson et al., 1999).
The purpose of the present study is to examine the
predictive ability of baseline neuropsychological performance
and clinical symptoms on the vocational
outcomes of a group of schizophrenia spectrum clients
enrolled in vocational services who were participating
in a longitudinal study comparing the effects of
atypical neuroleptic medications with traditional medications.
In addition, we examined both employment
outcomes (e.g., hours worked) as well as research staff
ratings of behaviors. We hypothesized that better
clinical symptom control and neuropsychological performance
at baseline would be correlated with and
predictive of better vocational outcomes 4 months
later.
2. Method
2.1. Subjects
The sample consisted of 112 clients recruited as
part of a longitudinal study to examine the effects of
atypical antipsychotic medications on vocational rehabilitation
outcomes (Bond et al., in press). Inclusion
criteria for the study were as follows: (1) diagnosis of
schizophrenia spectrum disorders based on DSM-IV
criteria and confirmed with the Structured Clinical
Interview for DSM-IV Axis I Disorders (SCID-I,
Research Version); (2) stable phase of disorder as
defined as having no hospitalizations, changes in
housing, or changes in medication within the past
month; (3) no diagnosis of mental retardation, neurological
disorder, or insult to the brain that might affect
their cognitive performance; (4) unemployed at time
of program admission; (5) between the ages of 18 and
64; (6) expressed a goal of paid employment and
currently prescribed an antipsychotic medication. On
average, participants were 40 years old, had 12.6
years of education (range 6–20), had 7.0 lifetime
psychiatric hospitalizations (S.D. = 10.7), and were
ill for a mean of 16.5 years (S.D. = 10.4). Most
participants were male (60.7%). The sample consisted
of primarily African-American (44.0%) and Caucasian
(52.0%) clients.
All clients were prescribed one or more antipsychotic
medications, including 14.0% on a traditional
antipsychotic and 54.2% were receiving an atypical
neuroleptic and 31.8% receiving a combination. There
were generally no associations between type of medication
and any of the predictor or outcome measures,
so medications were not used as a statistical control in
the current study.
2.2. Procedures
Study participants were recruited from among
new admissions to day programs at five sites of
Thresholds in Chicago (N = 83), and four community
mental health centers in Indianapolis (N = 29) from
March 1999 to January 2001. Thresholds is a psychiatric
rehabilitation agency providing intensive
vocational services in a stepwise manner, including
unpaid work crews, group and individual placements
arranged between Thresholds and employers,
enclaves, agency-run businesses, and independent
jobs. This vocational model has been dubbed the
‘‘diversified placement approach’’ (DPA) (Koop et
al., in press). The Indiana programs used an individual
placement model of supported employment,
although not strictly adhering to all the principles
of evidence-based supported employment (Bond et
al., 2001). A minority (N = 18) of study participants
from Thresholds also received individual placement
supported employment in lieu of the stepwise vocational
services. Because of the differences in the
program philosophies, the study sample was divided
into two subsamples: DPA (N = 65) and supported
employment (N = 47) for some analyses.
Clients giving informed consent were enrolled in
the study within 120 days of admission to Thresholds
or to the Indianapolis programs. Diagnosis was confirmed
using either the Structured Clinical Interview
for DSM-IV (SCID-I) (First et al., 1995) or the

334
J.D. Evans et al. / Schizophrenia Research 70 (2004) 331–342
computerized version of the Composite International
Diagnostic Interview (CIDI-A) (World Health Organization,
1997a,b). Participants then completed a
comprehensive evaluation that included an examination
of clinical symptoms, neuropsychological performance,
side effect rating scales, and ratings of
vocational performance. For the current study, client
vocational outcomes were examined 4 months after
the initial baseline assessments. Interviewers completing
the diagnostic assessments, clinical ratings, and
neuropsychological assessments were trained and supervised
by licensed clinical psychologists (JDE and
PHL) familiar with the instruments.
2.3. Measures
Clinical ratings of symptoms: The Positive and
Negative Syndrome Scale (PANSS; Kay et al.,
1987) is a 30-item rating scale completed by trained
research staff. Scores for positive, negative, cognitive
disorganization, emotional discomfort, and hostility/
excitement symptoms were derived from factor-derived
scoring published by Bell et al. (1994). Internal
consistency ratings ranged from a = 0.40 for hostility
symptoms to 0.73 for negative symptoms. The
PANSS total score a = 0.79.
Neuropsychological evaluation: All participants
completed a brief neuropsychological evaluation.
The following is a listing of neuropsychological
measures grouped by the cognitive domain that each
is intended to measure.
Learning: California Verbal Learning Test (CVLT;
Delis et al., 1987; Norman et al., 2000) Monday total
trials 1–5.
Secondary verbal memory: CVLT short-delay free
recall and long-delay free recall; percent retention
((long-delay free recall raw score/Monday trial
5 raw score) 100).
Executive functioning: Wisconsin Card Sorting
Test—perseverative responses, and categories correct
(WCST; Heaton et al., 1993), and Part B of the
Trailmaking Test (Reitan and Wolfson, 1993).
Short-term memory: WAIS-III Digit Span subtest
(Wechsler, 1997).
Speed of information processing/psychomotor
speed: WAIS-III Digit Symbol, WAIS-III Symbol
Search subtests (Wechsler, 1997), and Part A of the
Trailmaking Test (Reitan and Wolfson, 1993).
Raw scores of the neuropsychological tests were
converted into age-corrected scaled scores with a
mean of 10 and a standard deviation of 3 (WAIS-III
scores) or demographically corrected T-scores with a
mean of 50 and a standard deviation of 10 (CVLT,
WCST, and Trailmaking test) scores (Heaton et al.,
1993; Norman et al., 2000).
Work performance: The Work Behavior Inventory
(WBI; Bryson et al., 1997) is a standardized work
performance assessment instrument specifically
designed for people with severe mental illness. The
instrument is a 36-item inventory with five subscales
and one global item: Cooperativeness, Work habits,
Work quality, Social skills, and Personal presentation.
Each scale comprises 7 items individually assessed on
a 5-point scale where ‘‘1’’ represents ‘‘consistently
inferior performance’’ and ‘‘5’’ represents ‘‘consistently
superior performance.’’ The range for each
subscale score ranges from 7 to 35. A total score is
calculated as the sum of the five subscale scores
(range of scores from 7 to 175). Participants in
appropriate work settings were rated by a research
assistant who was trained by one of the developers of
this scale (PHL) and who followed the author’s
ratings protocol, making direct observation of each
client’s behavior at the work site and conducting a
brief interview with their immediate supervisor. WBI
ratings were completed on only those clients in
employment and prevocational work settings in which
a research assistant could directly observe work behavior.
This requirement excluded some competitive
jobs. The WBI ratings were limited to the Thresholds
DPA sample. Mean substitution of the WBI scores
were used for missing data points (less than 5% of all
responses) in the following analyses. The WBI had
excellent internal consistency in this sample (WBI
total score a = 0.97; Social skills a = 0.89; Cooperativeness
a = 0.84; Work habits a = 0.93; Work quality
a = 0.96; Personal presentation a = 0.89).
Employment outcome measures: Employment status
at follow-up was coded according to a modified
version of the Work Placement Scale (Meyer et al.,
2002), consisting of a 5-point gradient: (5) integrated
employment, which included both competitive employment
(a regular community job, with non-disabled
coworkers, paying minimum wage), and
individual placement (agency-contracted community
job), (4) group placement: paid community job at a

J.D. Evans et al. / Schizophrenia Research 70 (2004) 331–342 335
site working alongside other clients, or an agency-run
business: paid work activity at a business owned and
run by the rehabilitation agency, (3) sheltered workshop/volunteer
work: paid work activity licensed by
the Department of Labor and paid on a piece-rate
basis, or unpaid work activity in a community employment
site, (2) prevocational training: unpaid
work training at the rehabilitation agency, and (1)
unemployed: not engaged in any vocational activity.
Higher scores reflect more independent employment
activity. Other employment outcome measures included
weeks worked, total hours worked, and total wages
earned during follow-up.
2.4. Statistical methods
Prior to analyses, data were examined for normality
and homogeneity of variance. The WBI outcome
data met parametric assumptions. Violations of normality
were noted in the PANSS negative, cognitive,
and hostility factor scores, total wages earned, total
hours and total weeks worked, as well as the WAIS-III
digit symbol subtest scores indicating moderate positive
skew in these variables. However, parametric
tests have been shown to be robust to violations of
non-normality; therefore, all analyses used parametric
tests on untransformed variables (Linton and Gallo,
1975; Tabachnick and Fidell, 1996). Nevertheless, to
confirm the robustness of the findings, we also conducted
appropriate nonparametric tests (e.g., Spearman
rank order correlations) and found a similar
pattern of results. Employment status was examined
through two dichotomies: integrated employment vs.
other and supported employment (SE) vs. diversified
placement (DPA). This was done due to the different
emphasis placed on approaches to job placement in
the two programs. We examined all of the predictor
and outcome variables by program (supported employment
vs. diversified placement), and conducted
all of the hypothesized analyses by program type. T-
tests were conducted to compare the baseline clinical
and cognitive predictors of those achieving integrated
vs. nonintegrated employment. In addition, a series of
multivariate analyses of variance (MANOVAs) were
run on the baseline demographic, clinical, and neuropsychological
variables to determine if there were any
differences by program type. In addition, Pearson
product moment correlations between the dependent
variables (objective vocational data and WBI scores)
and the independent variables (clinical rating scales
and neuropsychological scores) were computed for
the two employment programs. Independent variables
having significant or near-significant ( p < 0.2) correlations
were entered into multiple regression analyses
to determine if the relationships persisted after simultaneous
adjustment (Dales and Ury, 1978). An alpha
level of 0.05(two-tailed) was used to define significance
for all analyses.
3. Results
3.1. Baseline characteristics
Among demographic variables, there were no significant
differences between the two groups in terms
of age, education, or duration of illness. Both groups
were about 39 years of age, had been ill about 16
years, and completed about 12 years of school.
There were also no differences in clinical or
neuropsychological variables between the Thresholds
program and the Indiana programs at baseline
(Table 1). The clinical ratings of illness at baseline
suggest that the participants had mild levels of
symptoms, as measured by the PANSS. In terms
of cognitive performances at baseline, the sample
was mildly to moderately impaired across the various
domains, (i.e., T-scores < 40). Specific deficits
were seen in verbal learning and memory performances
on the CVLT. Speed of information processing
was also moderately impaired compared to
published normative data (Wechsler, 1997). There
was a fair degree of variability in all symptom and
cognitive measures within the population, providing
good potential to detect how baseline differences
might affect outcomes.
3.2. Vocational outcomes
Four months into the study, an examination of the
WPS found that there were significant differences in
work placements as a function of employment program.
(See Table 2). The overall chi-square was
significant for WPS placement by employment program
(v 2 (4) = 37.4, p < 0.001). This is probably a
consequence of the diversified placement approach

336
J.D. Evans et al. / Schizophrenia Research 70 (2004) 331–342
Table 1
Clinical and neuropsychological performance at baseline
SE (N = 47)
(M F S.D.)
(DPA N = 65),
(M F S.D.)
PANSS subscales
Positive 2.3 F 0.9 2.5 + 1.0
Negative 1.6 F 0.5 1.9 + 0.7
Cognitive 1.9 F 0.6 1.9 + 0.7
Emotional discomfort 2.4 F 1.1 2.7 + 0.9
Hostility/excitement 1.3 F 0.5 1.3 + 0.4
PANSS total score 56.3 F 12.0 60.9 + 14.4
Neuropsychological tests
CVLT
List A total T-score 28.9 F 9.9 29.4 + 9.3
Short-delay free T-score 33.1 F 10.6 34.3 + 10.3
Long-delay free T-score 33.2 F 9.8 33.4 + 9.7
% retention score 79.8 F 36.9 82.1 + 32.7
WCST perseverative
30.5 F 11.9 31.5 + 11.4
response T-score
WCST categories correct 2.8 F 2.3 3.1 + 2.1
Trails A T-score 38.9 F 9.8 35.9 + 11.6
Trails B T-score 35.1 F 14.1 33.9 + 10.8
Symbol search (SS) 6.7 F 2.8 5.8 + 2.9
Digit symbol (SS) 5.9 F 2.0 5.6 + 2.1
Digit span (SS) 8.2 F 2.5 8.5 + 2.9
PANSS scores range from 1 –absent to 7 – extreme; neuropsychological
T-scores have a mean of 50 and a 10-point standard
deviation.; SS age-corrected scaled scores have a mean of 10 and a
3-point standard deviation.
PANSS = Positive and Negative Syndrome Scale; CVLT = California
Verbal Learning Test; WCST = Wisconsin Card Sorting Test;
SS = age-corrected scaled scores.
to employment emphasized at Thresholds. No significant
differences were noted in hours, weeks, or wages
earned among those clients working in the two programs.
As a group, those clients employed at 4
months worked an average of 19 h/week, and the
median hourly wage was $6.38 (range $0.15–$9.35/
h). The average number of paid weeks worked was
8.15 (range 0–39), with total paid wages averaging
$890 (range 0–$5661) during the 4-month interval.
In terms of those participants in integrated vs.
nonintegrated placement, no significant differences
were found in baseline demographic or clinical
domains. In terms of neuropsychological performance,
however, those in integrated work settings
had significantly better CVLT short-delay free recall
(37.2 vs. 32.9, t(110) = 1.98, p = 0.05) and CVLT
long-delay free recall (37.2 vs. 32.2, t(110) = 2.16,
p = 0.033) at baseline, compared to all other clients.
An examination by employment program found that
integrated employment in the DPA programs was
associated with better education (12.5 vs. 14.7,
t(63) = 2.4, p = 0.02); lower hostility symptoms
(1.36 vs. 1.0, t(55) = 6.3, p = 0.000); better CVLT
Monday total recall (28.8 vs. 35, t(63) = 1.89, p =
0.06); better CVLT short-delay recall (32.7 vs. 44.0,
t(63) = 3.39, p = 0.001); and CVLT long-delay free
recall (32.2 vs. 40.6, t(63) = 2.51, p = 0.015). However,
none of these differences were significant in the
supported employment programs.
Work performance ratings were obtained for 49
(75.3%) of 65 clients who were receiving vocational
services at 4 months in the DPA program (see Table
3). These 49 participants were in the following placements:
7 were in integrated placements, 15 were in
group placements, and 27 were in sheltered workshops
or prevocational programs at Thresholds. There
were no significant differences in WBI performance
as a function of work level. Furthermore, among these
clients, their work performance on the job was found
to be consistent with published norms (e.g., within 1
standard deviation) suggesting moderate to good work
performance (Bryson et al., 1997).
3.3. Correlates of work performance
Given the differences in work level as a function
of placement, we analyzed the correlates of performance
by employment program. We first looked at
three demographic variables. Of these variables,
only the following reached significance. For those
clients in supported employment, age was positively
related to weeks worked (r = 0.38). For the diversified
placement sample, duration of illness was
negatively associated with total paid wages (r =
0.25, p = 0.051).
Table 2
Work placement scale scores by employment program
Vocational level SE program DPA program
Integrated employment 12 (25.5%) 9 (13.8%)
Group placement 0 (0.0%) 17 (26.2%)
Sheltered workshops 3 (6.4%) 13 (20.0%)
Prevocational crews 7 (14.9%) 19 (29.2%)
Unemployed 25 (53.2%) 7 (10.8%)
SE = supported employment; DPA= diversified placement approach.

J.D. Evans et al. / Schizophrenia Research 70 (2004) 331–342 337
Table 3
WBI scores at 4 months
WBI subscales a
All clients
(N = 49)
(M F S.D.)
Integrated employment
(N =7) (M F S.D.)
Group placement
(N = 15) (M F S.D.)
Sheltered workshop
(N = 27) (M F S.D.)
Work habits 25.0 F 6.7 25.9 F 5.6 27.5 F 4.9 23.3 F 7.7
Personal presentation 25.7 F 5.3 26.3 F 2.2 26.7 F 4.9 25.0 F 6.0
Work quality 24.1 F 6.5 23.9 F 4.2 24.5 F 5.8 23.9 F 7.5
Social skills 21.6 F 5.7 23.0 F 4.4 22.7 F 4.2 21.0 F 6.5
Cooperativeness 24.8 F 4.8 25.4 F 3.2 25.3 F 4.8 24.3 F 5.5
WBI total score 121.2 F 25.2 124.4 F 17.1 126.7 F 20.8 117.6 F 29.6
WBI = Work Behavior Inventory.
a Range of subscales 7–35; higher scores = better performance.
Pearson correlations between the 4-month employment
outcomes and both the PANSS and neuropsychological
scores are presented in Table 4. Altogether,
5 of a possible 42 correlations were statistically significant
for the supported employment sample. There
was a trend for hostility symptoms on the PANSS to be
negatively associated with wages earned. In terms of
neuropsychological variables, retention on the CVLT
was positively associated with total hours worked, and
there was a trend for long-delay free recall to be related
to hours on the job, as well. There was also a modest
trend of worse performance on digit symbol being
negatively related to total weeks worked.
When examining the DPA group, however, a different
pattern of results was seen. A total of 9 of a possible
42 correlations were statistically significant. Specifically,
negative symptoms were inversely related to total
wages earned and total hours worked for this group.
Table 4
Correlations of 4-month work outcomes with baseline clinical and neuropsychological variables
SE N =47 DPA N =65
Total weeks
worked
Total wages
earned
Total hours
worked
Total weeks
worked
Total wages
earned
Total hours
worked
PANSS subscales
Positive 0.21 0.17 0.12 0.10 0.21 0.15
Negative 0.43** 0.16 0.09 0.24 # 0.31* 0.25*
Cognitive 0.05 0.07 0.16 0.09 0.18 0.14
Emotional 0.18 0.18 0.03 0.11 0.09 0.04
Hostility 0.21 0.28 # 0.23 0.20 0.08 0.21
PANSS total 0.01 0.15 0.10 0.21 0.30* 0.24 #
List A total T-score 0.11 0.12 0.10 0.27* 0.14 0.23 #
Short-delay free T-score 0.06 0.05 0.10 0.24 # 0.11 0.27*
Long-delay free T-score 0.04 0.10 0.28 # 0.17 0.11 0.17
CVLT % retention score 0.09 0.25 0.30* 0.03 0.04 0.04
WCST T-score 0.04 0.13 0.02 0.03 0.08 0.07
WCST categories 0.04 0.22 0.04 0.12 0.09 0.10
Trails A T-score 0.21 0.06 0.11 0.04 0.01 0.08
Trails B T-score 0.06 0.08 0.12 0.19 0.13 0.11
Symbol search (SS) 0.19 0.08 0.06 0.04 0.06 0.00
Digit symbol (SS) 0.30 # 0.17 0.04 0.12 0.20 0.08
Digit span (SS) 0.05 0.07 0.05 0.04 0.01 0.03
SE = supported employment; DPA= diversified placement approach; PANSS = Positive and Negative Syndrome Scale; CVLT = California
Verbal Learning Test; WCST = Wisconsin Card Sorting Test; SS = age-corrected scaled scores.
* p < 0.05.
**p < 0.01.
# trend p V 0.06.

338
J.D. Evans et al. / Schizophrenia Research 70 (2004) 331–342
Table 5
Correlations between WBI scores and both clinical symptoms and neuropsychological variables (N = 49)
Social skills Cooperativeness Work
habits
Work
quality
Personal
presentation
WBI total
PANSS subscales
Positive 0.02 0.07 0.08 0.04 0.13 0.04
Negative 0.24 0.12 0.07 0.16 0.08 0.12
Cognitive 0.38** 0.37** 0.15 0.34* 0.07 0.30*
Emotional 0.03 0.01 0.01 0.02 0.06 0.00
Hostility 0.16 0.26 0.10 0.01 0.02 0.11
PANSS total 0.25 0.24 0.06 0.16 0.08 0.14
List A total T-score 0.26 0.35* 0.38** 0.25 0.20 0.33*
Short-delay free T-score 0.12 0.30* 0.37** 0.24 0.18 0.29*
Long-delay free T-score 0.13 0.34* 0.40** 0.32* 0.28* 0.34*
CVLT % retention 0.04 0.17 0.26 0.24 0.36** 0.25
WCST T-score 0.16 0.06 0.13 0.06 0.29* 0.16
WCST categories 0.19 0.06 0.06 0.10 0.18 0.13
Trails A T-score 0.04 0.20 0.22 0.13 0.17 0.17
Trails B T-score 0.04 0.12 0.09 0.04 0.00 0.07
Symbol search (SS) 0.04 0.28* 0.28 # 0.21 0.12 0.22
Digit symbol (SS) 0.02 0.11 0.15 0.09 0.04 0.10
Digit span (SS) 0.14 0.25 0.11 0.17 0.09 0.17
WBI = Work Behavior Inventory; PANSS = Positive and Negative Syndrome Scale; CVLT = California Verbal Learning Test; WCST = Wisconsin
Card Sorting Test; SS = age-corrected scaled scores.
**p < 0.01.
# non significant trend, p-value = 0.054.
There was also a trend for more negative symptoms to
be associated with fewer weeks on the job. In addition,
the PANSS total score was associated with total wages
earned. In terms of cognitive performance, both learning
and short-delay free recall on the CVLT were
significantly related to total weeks worked. There
was also evidence for both of these measures to be
related to total hours on the job as well.
Within the DPA sample, the correlations of both
clinical symptom scores and neuropsychological variables
with the WBI subscales are presented in Table
5. The cognitive subscale of the PANSS was significantly
related to the social skills, cooperativeness,
and work quality subscales of the WBI, as well as the
WBI total score. In addition, perseverative responses,
learning and memory, and speed of information processing
were related to performance on the WBI (see
Table 5). The WBI scores at 4 months were not related
to hours, weeks, or wages earned in this sample.
3.4. Prediction of WBI scores and occupational status
In order to determine the relative importance of the
clinical and neuropsychological variables, we conducted
a multiple regression analysis with the total
WBI score as the dependent variable (see Table 6). We
used a stepwise analysis with the following variables
as predictors in the analysis: PANSS cognitive and
negative subscale scores, learning and memory variables
(Monday total T, both short-delay and longdelay
free recall T-scores; % retention), perseverative
Table 6
Prediction of WBI total score at 4 months
Variables to enter Beta Step 2 beta
CVLT long-delay T score 0.36* 0.37*
PANSS cognitive score 0.35*
R 2 0.132 0.257
R 2 change 0.125
F(1,42) 6.39*
F(2,41) 7.09**
N = 44.
WBI = Work Behavior Inventory; PANSS = Positive and Negative
Syndrome Scale; CVLT = California Verbal Learning Test;
WCST = Wisconsin Card Sorting Test; SS = age-corrected scaled
scores.
* p < 0.05.
**p < 0.01.

J.D. Evans et al. / Schizophrenia Research 70 (2004) 331–342 339
responses from the WCST, and a composite processing
speed variable (mean of WAIS Digit Symbol,
Symbol Search, and Trails A scores). This found a
significant model ( F(2,41) = 7.09, p = 0.002), accounting
for 22% of the variance, with the cognitive
subscale of the PANSS and the CVLT long-delay free
recall score as the only significant predictors of WBI
total score performance 4 months later. We then ran
stepwise analyses using the CVLT variables, the
processing speed composite score and PANSS
cognitive subscales as predictors in each of the
WBI subscales.
In the stepwise analyses, significant results were
seen in the WBI work quality, social skills, work
habits, and cooperativeness subscales. The multiple
Rs ranged from 0.44 (work habits) to 0.53 (cooperativeness)
accounting for 18–25% of the variance in
these functional parameters. Examination of the individual
weights among tests suggested differential
contributions of cognitive abilities and clinical symptoms
in each of these domains. Specifically, both
PANSS cognitive disorganization symptoms and
long-delayed recall predicted uncooperativeness and
poor work quality. PANSS cognitive disorganization
symptoms were the only significant predictor of poor
social skills and poor work habits.
Given these findings, we next tried to predict
employment outcomes using significant baseline demographic,
clinical, and neuropsychological variables.
We used the integrated vs. nonintegrated
employment categories and ran a logistic regression
analysis to predict group membership (N = 96). We
entered predictors in blocks, with age, duration of
illness, education, and gender entered first, followed
by PANSS negative and cognitive symptoms in the
second block, and finally the CVLT measures, processing
speed composite score, and WCST perseverative
response T-score. The overall model was
significant (v 2 = 23.7, p = 0.014), and the significant
predictors in the final equation included education
(Wald = 5.44, df =1, p = 0.020), the learning index
(Monday total trials 1–5 T-score) from the CVLT
(Wald = 5.32, df =1,p = 0.021), and CVLT long-delay
free recall (Wald =4.60, df =1, p = 0.032 ). However,
the improvement in classification rate was modest
(from 80.2% at step 1 to 81.3% at step 3). None of
the clinical symptoms as assessed by the PANSS were
significant in the model.
4. Discussion
The results of the present study found modest
evidence for the relationship of better neuropsychological
performance being predictive of vocational
outcomes among clients with schizophrenia receiving
comprehensive vocational rehabilitation services. We
found that among aspects of work behavior, verbal
learning and memory performance, speed of information
processing, executive functioning, and cognitive
disorganization symptoms were predictive of better
employment outcomes. In addition, better learning
and memory performances were predictive of objective
measures of work status, namely, integrated vs.
nonintegrated employment. Furthermore, among both
programs, aspects of learning and memory were
associated with hours, weeks, and wages earned.
The most interesting finding, however, was that these
relationships were more prominent in the diversified
placement approach than compared to the supported
employment program.
We found a significant difference in baseline neuropsychological
performance on the CVLT as a function
of work outcome. Those people who achieved
employment in integrated placements had performed
slightly better on this measure of cognitive performance
than those who were not in integrated settings.
However, we did not find any other significant differences
in baseline clinical or neuropsychological domains
as a function of employment status. This is
consistent with the pattern of findings in two recent
studies involving clients enrolled in vocational programs
(Gold et al., 2002; McGurk et al., 2003). We
speculate that clients in the current study benefited
from the structure and support of the vocational
services to the extent that they were successful in
starting some form of paid employment despite their
level of cognitive impairment, but their actual performance
on the job was more directly affected by
cognitive impairments.
The only relationships between the neuropsychological
measures and employment outcome were seen
in the domains of learning and memory, processing
speed, and executive function. Other studies have
found that multiple aspects of cognition were related
to improvements in work behaviors (Bell and Bryson,
2001; Brekke et al., 1997; Green and Nuechterlein,
1999; McGurk et al., 2003). Specficially, much of the

340
J.D. Evans et al. / Schizophrenia Research 70 (2004) 331–342
prior work found a relationship of executive functioning
and verbal learning and memory to aspects of
vocational outcome (McGurk, 2000; McGurk et al.,
2003).
In terms of clinical correlates of vocational outcome,
negative symptoms were related to total weeks
worked, total hours, and total wages earned, and
hostility symptoms were modestly related to total
wages earned. This is consistent with previous work
(McGurk and Meltzer, 2000). However, we did not
find that cognitive disorganization symptoms were
related to other measures of vocational status. Furthermore,
none of these clinical symptoms were
predictive of integrated employment status 4 months
later. This lack of a relationship was also seen in the
Bryson and Bell (2003) study.
Although the relationship of clinical and neuropsychological
variables to employment status was modest,
some relationships with work performance were
strong. We found that better verbal memory, improved
executive functioning, processing speed, and less
cognitive disorganization symptoms predicted aspects
of behaviors on the job. Other studies have also found
that neuropsychological performance was predictive
of these work behaviors (Bell and Bryson, 2001;
Bryson and Bell, 2003; Green et al., 2000; Lysaker
et al., 1995a).
The present findings suggest that cognitive abilities
may be important to employment outcomes. In addition,
the compensatory nature of the vocational rehabilitation
program may lessen or ameliorate the
negative impact of the cognitive deficits seen in
schizophrenia. It may be that once an individual
obtains a vocational placement, aspects of work
behavior determine job tenure and maintenance. Another
study found that the WBI was predictive of total
hours worked on the job 6 months after leaving a
vocational rehabilitation program (Bryson et al.,
1999). This suggests that the cognitive impairments
have their greatest impact on these work behaviors,
not on work status (Bryson and Bell, 2003; Gold et
al., 2002).
There are several limitations to the current study
that should be addressed. First, although this is a
predictive study, the length of follow-up is only 4
months. Thus, the chance to secure employment was
limited by setting issues and speed of placement. It
should be noted, however, that 66% of the sample
were engaged in some form of work activity, and at 4
months, 37% were in fact, in paid employment settings.
A second limitation is the problem of alpha
inflation and spurious correlation as a result of the
large number of correlations. A third limitation is the
restriction of range in symptoms, making it difficult to
find significant correlations between PANSS scores
and vocational outcomes. The study sample is representative,
however, of many stable, outpatient clients
living in the community.
In conclusion, this study found modest support
for a link between cognitive functioning and vocational
outcomes. This link was strongest between the
more proximal measures of work performance and
barely present when examining the correlations with
employment outcomes. Vocational services seem best
equipped to help people get jobs, but do not necessarily
help clients in the performance of their duties
on the jobs. Therefore, these programs may not
always succeed in helping clients maintain paid jobs
once they have obtained them. Consequently, for
clients with schizophrenia to succeed in the vocational
sphere, both elements are necessary: professional
support to find the jobs and the appropriate
work behaviors and skills necessary to maintain a
competitive level of employment.
Acknowledgements
This work was supported by a grant from Eli Lilly
and Company Pharmaceuticals. A preliminary version
of this research was presented at the 2002 Annual
Meeting of the International Neuropsychological
Society (Toronto, Canada). The authors wish to
acknowledge the contributions of Marc Lauritano
and Alan McGuire for their assistance with coding
employment data, in addition to the numerous
research assistants for their efforts in collection of
the data for this study.
References
Bell, M.D., Bryson, G., 2001. Work rehabilitation in schizophrenia:
does cognitive impairment limit improvement? Schizophrenia
Bulletin 27 (2), 269–279.
Bell, M.D., Lysaker, P.H., 1995. Psychiatric symptoms and work

J.D. Evans et al. / Schizophrenia Research 70 (2004) 331–342 341
performance among persons with severe mental illness. Psychiatric
Services 46 (5), 508–510.
Bell, M.D., Lysaker, P.H., Goulet, J.G., Milstein, R.M., Lindenmayer,
J.P., 1994. Five component model of schizophrenia: factorial
invariance of the positive and negative syndrome scale.
Psychiatry Research 52, 295–303.
Bellack, A.S., Gold, J.M., Buchanan, R.W., 1999. Cognitive rehabilitation
for schizophrenia: problems, prospects, and strategies.
Schizophrenia Bulletin 25 (2), 257–274.
Bond, G.R., Friedmeyer, M.H., 1987. Predictive validity of situational
assessment at a psychiatric rehabilitation center. Rehabilitation
Psychology 32, 99–112.
Bond, G.R., Becker, D.R., Drake, R.E., Rapp, C.A., Meisler, N.,
Lehman, A.F., et al., 2001. Implementing supported employment
as an evidence-based practice. Psychiatric Services 52
(3), 313–322.
Bond, G.R., Kim, H.-W., Meyer, P., Gibson, J., Tunis, S., Evans, J.,
et al., 2004. Response to vocational rehabilitation during treatment
with olanzapine, risperidone, or first-generation antipsychotics.
Psychiatric Services 55, 59–66.
Brekke, J.S., Raine, A., Ansel, M., Lencz, T., Bird, L., 1997. Neuropsychological
and psychophysiological correlates of psychosocial
functioning in schizophrenia. Schizophrenia Bulletin 23
(1), 19–28.
Bryson, G., Bell, M.D., 2003. Initial and final work performance in
schizophrenia: cognitive and symptom predictors. Journal of
Nervous and Mental Disease 191 (2), 87–92.
Bryson, G., Bell, M.D., Lysaker, P., Zito, W., 1997. The work
behavior inventory: a scale for the assessment of work behavior
for people with severe mental illness. Psychiatric Rehabilitation
Journal 20 (4), 47–55.
Bryson, G., Bell, M.D., Kaplan, E., Greig, T., 1998. The functional
consequences of memory impairments on initial work performance
in people with schizophrenia. Journal of Nervous and
Mental Disease 186 (10), 610–615.
Bryson, G., Bell, M.D., Greig, T., Kaplan, E., 1999. The work
behavior inventory: prediction of future work success of people
with schizophrenia. Psychiatric Rehabilitation Journal 23,
113 – 117.
Corrigan, P., Green, M., 1993. Schizophrenic patients’ sensitivity to
social cues: the role of abstraction. American Journal of Psychiatry
150, 589–594.
Dales, L.G., Ury, H.K., 1978. An improper use of statistical significance
testing in studying covariables. International Journal of
Epidemiology 7 (4), 373–375.
Delis, D.C., Kramer, J.H., Kaplan, E., Ober, B.A., 1987. California
Verbal Learning Test (CVLT) manual. The Psychological Corporation,
New York.
First, M.B., Spitzer, R.L., Gibbon, M., Williams, J.B.W., 1995.
Structured Clinical Interview for DSM-IV Axis I Disorder-Patient
Edition (SCID-I/P). Biometrics Research Department, New
York State Psychiatric Institute, New York.
Gold, J.M., Goldberg, R.W., McNary, S.W., Dixon, L.B., Lehman,
A.F., 2002. Cognitive correlates of job tenure among
patients with severe mental illness. American Journal of Psychiatry
159 (8), 1395–1402.
Green, M.F., 1996. What are the functional consequences of neurocognitive
deficits in schizophrenia? American Journal of Psychiatry
153 (3), 321–330.
Green, M.F., Nuechterlein, K.H., 1999. Should schizophrenia be
treated as a neurocognitive disorder? Schizophrenia Bulletin
25 (2), 309–318.
Green, M.F., Kern, R.S., Braff, D.L., Mintz, J., 2000. Neurocognitive
deficits and functional outcome in schizophrenia: Are we
measuring the ‘‘right stuff’’? Schizophrenia Bulletin 26 (1),
119–136.
Harvey, P.D., Howanitz, E., Parrella, M., White, L., Davidson, M.,
Mohs, R.C., et al., 1998. Symptoms, cognitive functioning, and
adaptive skills in geriatric patients with lifelong schizophrenia: a
comparison across treatment sites. American Journal of Psychiatry
155 (8), 1080– 1086.
Heaton, R.K., Chelune, G.J., Talley, J.L., Kay, G.G., Curtiss, G.,
1993. Wisconsin Card Sorting Test manual: revised and expanded.
Psychological Assessment Resources, Odessa.
Kay, S.R., Fiszbein, A., Opler, L.A., 1987. The positive and negative
syndrome scale (PANSS) for schizophrenia. Schizophrenia
Bulletin 13 (2), 261–276.
Koop, J., Rollins, A.L., Bond, G.R., Salyers, M.P., Dincin, J., Kinley,
T., et al., in press. Development of the DPA Fidelity Scale:
Using fidelity to define an existing vocational model. Psychiatric
Rehabilitation Journal.
Linton, M., Gallo, P.S., 1975. The practical statistician: simplified
handbook of statistics. Brooks/Cole, Monterey.
Lysaker, P., Bell, M., Beam-Goulet, J., 1995a. Wisconsin Card
Sorting test and work performance in schizophrenia. Psychiatry
Research 56 (1), 45–51.
Lysaker, P.H., Bell, M.D., Zito, W.S., Bioty, S.M., 1995b. Social
skills at work. Deficits and predictors of improvement in
schizophrenia. Journal of Nervous and Mental Disease 183
(11), 688–692.
McGurk, S.R., 2000. Neurocognition as a determinant of employment
status in schizophrenia. Journal of Psychiatric Practice 10
(July), 190–196.
McGurk, S.R., Meltzer, H.Y., 2000. The role of cognition in vocational
functioning in schizophrenia. Schizophrenia Research 45
(3), 175–184.
McGurk, S.R., Mueser, K.T., Harvey, P.D., La Puglia, R.,
Marder, J., Davis, K.L., 2003. Cognitive and clinical predictors
of work outcomes in clients with schizophrenia receiving
supported employment services. Psychiatric Services 54 (8),
1129–1135.
Meyer, P.S., Bond, G.R., Tunis, S.L., McCoy, M.L., 2002. Comparison
between the effects of atypical and traditional antipsychotics
on work status for clients in a psychiatric rehabilitation
program. Journal of Clinical Psychiatry 63, 108–116.
Norman, M.A., Evans, J.D., Miller, S.W., Heaton, R.K., 2000. Demographically
corrected norms for the California Verbal Learning
Test. Journal of Clinical and Experimental Neuropsychology
22, 80–94.
Patterson, T.L., Semple, S.J., Shaw, W.S., Halpain, M., Moscona,
S., Grant, I., et al., 1997. Self-reported social functioning
among older patients with schizophrenia. Schizophrenia Research
27 (2–3), 199–210.
Reitan, R.M., Wolfson, D., 1993. The Halstead–Reitan Neuropsy-

342
J.D. Evans et al. / Schizophrenia Research 70 (2004) 331–342
chological Test Battery, 2nd ed. Neuropsychology Press, Tuscon,
AZ.
Smith, T.E., Hull, J.W., Romanelli, S., Fertuck, E., Weiss, K.A.,
1999. Symptoms and neurocognition as rate limiters in skills
training for psychotic patients. American Journal of Psychiatry
156 (11), 1817–1818.
Tabachnick, B.G., Fidell, L.S., 1996. Using Multivariate Statistics,
3rd ed. Harper Collins, New York.
Velligan, D.I., Bow-Thomas, C., Mahurin, R.K., Miller, A.L., Halgunseth,
L.C., 2000a. Do specific neurocognitive deficits predict
specific domains of community function in schizophrenia? Journal
of Nervous and Mental Disease 188 (8), 518–524.
Velligan, D.I., Bow-Thomas, C.C., Huntzinger, C., Ritch, J., Ledbetter,
N., Prihoda, T.J., et al., 2000b. Randomized controlled
trial of the use of compensatory strategies to enhance adaptive
functioning in outpatients with schizophrenia (comment). American
Journal of Psychiatry 157 (8), 1317–1323.
Wechsler, D., 1997. Manual for the Wechsler Adult Intelligence
Scale-III. The Psychological Corporation, San Antonio.
World Health Organization, 1997a. CIDI-Auto Version 2.1: Administrator’s
Guide and Reference. Training and Reference Centre
for WHO CIDI, Sydney.
World Health Organization, 1997b. Composite International Diagnostic
Interview (CIDI)—Core Version 2.1. Training and Reference
Centre for WHO CIDI, St. Louis.