Is headspace making a difference to young people’s lives?

2. Evaluation Methodology
Table 2.3 Description of outcomes variables used for DID analysis
Outcome variable
Description
2. Sexual practices and
risk in gay men
Psychological distress
Incapacity
Social Inclusion
Physical health
Binge drinking
Cannabis use
One of the evaluation’s central outcomes of interest is the mental health of clients.
Psychological distress is widely used as an indicator of mental health. For this evaluation,
psychological distress is measured using the Kessler Psychological Distress Scale
(K10). The K10 is a 10-item questionnaire that asks respondents how frequently they
have experienced symptoms of psychological distress during the past 4 weeks. For each
question, there is a five-level response scale based on how frequently the respondent
has experienced particular feelings (from ‘none of the time’ to ‘all of the time’). Each of the
items are scored from 1 (none of the time) to 5 (all of the time). Scores for the 10 items are
then added with total scores ranging between 10 and 50. Low scores indicate low levels
of psychological distress and high scores indicate high levels of psychological distress.
There are a number of cut offs developed for analysis of K10 scores. For the analysis
conducted for the evaluation we draw on the categorisations used in the Australian Bureau
of Statistics health surveys. The cut offs used in this analysis are: low (10-15), moderate
(16-21), high (22-29) and very high (30-50).
The number of days incapacitated because of psychological distress was gauged based on
two questions: ‘in the last 4 weeks, how many days were you totally unable to work, study,
or manage your day-to-day activities because of these feelings’ and ‘aside from those days
in the last 4 weeks, how many days were you able to work or study or manage your day to
day activities, but had to cut down on what you did because of these feelings’. Responses
to these questions are analysed and are referred to as days out of role (DOR) and days cut
back (DCB), respectively.
Individuals’ feelings of being socially supported, or included, was gauged using the
question ‘do you feel like there are people who are there for you?’ with responses recorded
on a 5-point scale from ‘never’ to ‘all of the time’. The proportion of young people that
responded they felt like people were there for them ‘all of the time’ or ‘most of the time’
were assessed across waves.
Measure assesses the frequency in which physical health problems have been the primary
cause of psychological distress in the last four weeks. This measurement was based on
responses to the question ‘In the last four weeks, how often have physical health problems
been the main cause of these feelings?’ and was rated on a 5-point scale from ‘none of the
time’ to ‘all of the time’.
Binge drinking is classified as excessive consumption of alcohol on a single occasion.
Binge drinking was gauged using responses to ‘During the last 30 days, on how many
days did you have 4 or more standard drinks of alcohol in a row, that is, within a couple
of hours?’. Responses were recorded on a scale from ‘0 days’ to ’20 or more days’ and a
midpoint mean calculation was derived. This question was answered only by those young
people who respond that they have had a drink of alcohol in the last 30 days.
Cannabis consumption relates to whether or not the young person has used cannabis
during the last 30 days. This question was only answered by young people who responded
that they have ever tried cannabis/marijuana.
2.4.2 The Clinically Significant Change Method
The evaluation was originally designed to assess young people’s outcomes only through the analysis
of survey data as the administrative dataset included in the first evaluation contained very limited
outcomes data. However, the new dataset launched at the beginning of the evaluation contains
robust outcomes data. The evaluators employed the Clinically Significant Change (CSC) method
to analyse these data. This method has been used to assess meaningful clinical changes in
psychological distress as measured by the Kessler 10 scale (K10) for all headspace clients captured
in the hCSA in order to examine the factors related to improvement and deterioration following
treatment. The CSC method also utilises the comparison surveys as a source to derive a functional
population, from which to compare changes in functioning of the ‘headspace treatment’ group over
time (Jacobson &Truax, 1991; Bauer et al, 2004; Atkins et al, 2005).
The CSC method groups the changes in K10 scores for all young people on a seven-point scale that
indicates improvement or deterioration in mental health functioning (clinically significant improvement,
reliably significantly improvement, insignificant improvement, no change, insignificant decline, reliably
significant decline, and clinically significant decline). Two measures of change are the most important:
– a reliable change and a clinically significant change. A reliable change (RC) in K10 scores between
two occasions of service is one that represents a statistically significant improvement (if the change
corresponds to a reduction in K10) or deterioration (if the change is an increase in K10) relative to an
Social Policy Research Centre 2015
headspace Evaluation Final Report
15