Is headspace making a difference to young people’s lives?
Evaluation-of-headspace-program
Evaluation-of-headspace-program
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
2. Evaluation Methodology<br />
Name Source type/description Dates<br />
Survey of <strong>young</strong> people<br />
The evaluation drew upon data collected as<br />
part of the National Child and Adolescent<br />
Survey of Mental Health and Wellbeing. A<br />
sample of this data comprises the W1 12-17<br />
years comparison group data. Wave 2 12-17<br />
years comparison group data; all 18-25 years<br />
comparison group data; and <strong>headspace</strong><br />
treatment group data collected for evaluation<br />
Wave 1 YMM data collection period:<br />
Jun 2013 <strong>to</strong> Mar 2014. Wave 2 data<br />
collection period: May 2014 <strong>to</strong> Nov 2014<br />
Wave 1 18-25 years comparison group<br />
data collection period: Dec 2013. Wave<br />
2 data collection period: Sep 2014 <strong>to</strong><br />
Oct 2014<br />
Wave 1 intervention group data<br />
collection period: Dec 2013 <strong>to</strong> Jun 2014.<br />
Wave 2 data collection period: Aug<br />
2014 <strong>to</strong> Jan 2015<br />
Interviews with <strong>headspace</strong> staff Primary data collected for evaluation Fieldwork conducted Apr <strong>to</strong> Jun 2013<br />
Interviews with <strong>headspace</strong> clients Primary data collected for evaluation Fieldwork conducted Apr <strong>to</strong> Jun 2013<br />
Interviews with parents and carers Primary data collected for evaluation Fieldwork conducted Aug <strong>to</strong> Oct 2014<br />
Survey of parents and carers Primary data collected for evaluation Online survey open Jun <strong>to</strong> Sep 2014<br />
Survey of centre managers Primary data collected for evaluation Survey open Jun <strong>to</strong> Aug 2013<br />
Professional stakeholders survey Primary data collected for evaluation Online survey open Sep <strong>to</strong> Nov 2014<br />
2.3 Evaluation methods<br />
The multiple evaluation methods enable a comprehensive assessment of <strong>headspace</strong> effectiveness<br />
and efficiency. Where possible, the different methods were triangulated <strong>to</strong> strengthen the findings<br />
and <strong>to</strong> provide more depth <strong>to</strong> the analysis. The evaluation methods are summarised in Table 2.2<br />
below. Further information on each method including sampling, recruitment and analysis techniques<br />
is provided in Appendix C.<br />
Table 2.2 Summary of evaluation methods<br />
Evaluation method<br />
Description of method<br />
Analysis of program data:<br />
<strong>headspace</strong> Centres Services<br />
Application (hCSA)<br />
Analysis of secondary data:<br />
(Census)<br />
Surveys of <strong>young</strong> people<br />
Analysis of the data contained in the <strong>headspace</strong> Centres Services Application (hCSA)<br />
for the 2013/14 financial year. This dataset contains information collected from both<br />
<strong>headspace</strong> clients and service providers. The hCSA data has been operationalised<br />
<strong>to</strong> collect information about <strong>young</strong> <strong>people’s</strong> access and engagement with <strong>headspace</strong>,<br />
aspects of the <strong>headspace</strong> service delivery model, patterns of client outcomes, government<br />
investment in <strong>headspace</strong>, and the extent of clinically significant change in <strong>headspace</strong><br />
clients. Descriptive analysis was undertaken <strong>to</strong> profile <strong>headspace</strong> clients in comparison <strong>to</strong><br />
the overall youth population. The hCSA data was also used <strong>to</strong> provide further information<br />
about the <strong>headspace</strong> survey intervention group.<br />
The evaluation was informed by the analysis of secondary data: the 2011 Census of<br />
Population and Housing. This secondary dataset was used <strong>to</strong> inform comparative analysis<br />
of the demographic characteristics of <strong>headspace</strong> clients and <strong>young</strong> people across the<br />
general population.<br />
Three surveys were conducted <strong>to</strong> assess <strong>young</strong> <strong>people’s</strong> outcomes for the <strong>headspace</strong><br />
evaluation. The surveys were conducted in two waves, approximately 9 months apart.<br />
However, this timeframe varied considerably.<br />
The three surveys undertaken were: the ‘<strong>headspace</strong> treatment’ group (n = 1,364 at wave<br />
2), a comparison group of 12-17 year olds (via the Young Minds Matter survey; n=1,686<br />
at wave 2), and a comparison group of 18-25 year olds (sourced through a national online<br />
panel; n=936 at wave 2).<br />
The survey questions were largely drawn from the Young Minds Matter survey.<br />
<strong>Is</strong>sues with survey timing and comparability are addressed in the analysis where possible<br />
and further information is provided in Appendix C.<br />
The three surveys were used <strong>to</strong> compare outcomes in <strong>young</strong> people that had sought<br />
services from <strong>headspace</strong> <strong>to</strong> those that had not sought any treatment and those that had<br />
sought other treatment from another mental health provider. A <strong>difference</strong>-in-<strong>difference</strong><br />
approach was used <strong>to</strong> assess the impact of the program using survey cohorts. Propensity<br />
score matching was conducted <strong>to</strong> align the intervention and comparison groups.<br />
Social Policy Research Centre 2015<br />
<strong>headspace</strong> Evaluation Final Report<br />
12