Is headspace making a difference to young people’s lives?

Recommendations

Info

Appendix C 6 June 2014. The ‘no treatment’ group is drawn from the comparison surveys and comprises young people who have not sought any substantial headspaces treatment and who also did not seek any treatment from any other health professionals between the two survey waves. The ‘other treatment’ group comprises of young persons within the comparison surveys and includes all those who sought support from a health professional(s) between Waves 1 and 2. A key limitation of this group is that no information about the dose of the other treatment is known. Young people within the comparison surveys that sought a substantive headspace treatment between Wave 1 and Wave 2 were excluded from the analysis. This constitutes 90 observations. Steps were also taken to match the headspace survey intervention group to administrative (hCSA) data in order to assess the timing of the survey collection against the period of treatment at a headspace centre. This process uncovered 340 observations (32.3%) that had completed their headspace treatment prior to the Wave 1 collection date. These observations were excluded from the analysis. Difference-in-difference Method The objective of the difference-in-difference (DID) method is to assess the changes in young people’s mental health, physical health, drug and alcohol use and social inclusion outcomes after using headspace services relative to other comparable young people that did not receive treatment at headspace. A difference-in-difference approach has been applied, using survey data for headspace treatment and comparison groups that have been matched on a number of different factors. Differencein-differences estimates are defined as the difference in the average outcome in the headspace treatment group at two points in data collection, that is wave 1 and wave 2, minus the difference in the average outcome in the matched comparison group. Comparison groups have been matched using a propensity score matching (PSM) technique and have been divided into groups that have received no mental health treatment between the survey waves and those that have received treatment other than headspace. Both are described further below. The differences in outcomes for headspace clients, for matched samples drawn from the no-treatment and other-treatment comparison groups, and for the difference in differences between headspace and either of these two matched comparison groups, are tested for their statistical significance and using effect size measures. Statistical significance is judged using an orthodox t-test for the mean differences between the headspace treatment group and a propensity-score matched comparison group, and for the difference in the differences between headspace and comparison groups. Effect sizes can be expressed in a number of ways, with Cohen’s d commonly reported as a standard indicator in clinical evaluation. The Cohen effect size measure presents a standardised difference in means across the course of an intervention (ie. the ratio of mean difference to a pooled standard deviation measure). In this regard it relates to classical measures of the significance of mean differences. It is common practice to interpret effect sizes according to the magnitude of the index – the normal ‘rule of thumb’ is to consider effects as small (0.2) through medium (0.5) to large (0.8) when relating the size of the effects. Propensity Score Matching A complexity of the DID methodology stems from the fact that those in headspace treatment group are often dissimilar in their observed and unobserved characteristics when compared to young people within the comparison surveys. In order to mitigate these differences, a propensity score matching technique has been applied to the datasets. The implementation of propensity score methods requires that a set of characteristics are chosen as the basis for alignment of the headspace treatment with a matched comparison group. It would seem appropriate to match simultaneously on a large number of factors, however, the match on each single characteristic becomes less precise as the number of matching characteristics increases. For the purpose of this evaluation, four characteristics were chosen - age, gender, K10 score and the number of non-functioning days ‘out of role’ to source a match between the headspace and comparison Social Policy Research Centre 2015 headspace Evaluation Final Report 178
Appendix C groups. The age and sex distributions below illustrate the closeness of the groups after undertaking propensity score matching. The majority of age groupings are closely aligned between the headspace treatment group and the matched no and other treatment groups. An exception are those aged 18-19 years. A higher representation of 18-19 year olds is present in the headspace treatment group when compared to the other matched treatment groups (17% compared to 12%). This difference is statistically significant. Alignment of the proportion of males and females between the groups is very precise after propensity score matching. It is recognised that the headspace treatment survey data has a greater representation of females when compared to the overall representation in the headspace administrative data (80% compare to 63%). This needs to be taken into account when seeking to generalise the results to the entire headspace clientele. Figure C7 Age and sex distributions of matched groups Source: Authors calculations from headspace evaluation survey data. As with other methods, PSM methods are also limited by an inability to account for unobserved differences between headspace clients and others with similar observed characteristics but who remain outside the headspace program. The PSM does improve to an extent, the identification of headspace effectiveness relative to matched comparison groups. Interviews with headspace clients and staff Sample and recruitment Qualitative interviews were conducted between April and June 2013 with 50 young people who were receiving headspace services, and 25 headspace staff from 5 of the 40 operational headspace centres around Australia (that is, 5 headspace staff and 10 young people per site). The fieldwork sites are not a representative sample of headspace locations around Australia. Site selection was based on a range of primary and secondary factors as outlined below. The primary factors focus on diversity, while the secondary factors aim to minimise the burden on participant sites Social Policy Research Centre 2015 headspace Evaluation Final Report 179
Page 1 and 2:
Is headspace making a difference to
Page 3 and 4:
Authors This report was written by
Page 5 and 6:
5. Service Delivery Model 73 5.1 Wh
Page 7 and 8:
Table 6.1 Table 6.2 Table 6.3 Main
Page 9 and 10:
Figure 4.9 Figure 4.10 Figure 4.11
Page 11 and 12:
Executive Summary headspace 1 aims
Page 13 and 14:
Executive Summary or very high leve
Page 15 and 16:
Executive Summary through mental he
Page 17 and 18:
value to society of the improvement
Page 19 and 20:
1. Introduction people in Australia
Page 21 and 22:
2. Evaluation Methodology 2.1 Aims
Page 23 and 24:
2. Evaluation Methodology Evaluatio
Page 25 and 26:
2. Evaluation Methodology Table 2.3
Page 27 and 28:
2. Evaluation Methodology Represent
Page 29 and 30:
3. Access and Engagement with Centr
Page 31 and 32:
Page 33 and 34:
Page 35 and 36:
Page 37 and 38:
Page 39 and 40:
Page 41 and 42:
Page 43 and 44:
Page 45 and 46:
Page 47 and 48:
Page 49 and 50:
Page 51 and 52:
4. Outcomes of headspace Clients Th
Page 53 and 54:
4. Outcomes of headspace Clients 35
Page 55 and 56:
4. Outcomes of headspace Clients Ch
Page 57 and 58:
4. Outcomes of headspace Clients No
Page 59 and 60:
4. Outcomes of headspace Clients Th
Page 61 and 62:
4. Outcomes of headspace Clients Fi
Page 63 and 64:
4. Outcomes of headspace Clients A
Page 65 and 66:
4. Outcomes of headspace Clients Ta
Page 67 and 68:
4. Outcomes of headspace Clients nu
Page 69 and 70:
4. Outcomes of headspace Clients No
Page 71 and 72:
4. Outcomes of headspace Clients Ta
Page 73 and 74:
y three percentage points, and 2.1
Page 75 and 76:
4. Outcomes of headspace Clients So
Page 77 and 78:
4. Outcomes of headspace Clients I
Page 79 and 80:
4. Outcomes of headspace Clients in
Page 81 and 82:
4. Outcomes of headspace Clients Wh
Page 83 and 84:
5. Service Delivery Model For the a
Page 85 and 86:
5. Service Delivery Model the provi
Page 87 and 88:
5. Service Delivery Model headspace
Page 89 and 90:
5. Service Delivery Model services,
Page 91 and 92:
5. Service Delivery Model services
Page 93 and 94:
5. Service Delivery Model Further,
Page 95 and 96:
5. Service Delivery Model that we
Page 97 and 98:
5. Service Delivery Model is a cons
Page 99 and 100:
5. Service Delivery Model connectio
Page 101 and 102:
5. Service Delivery Model Table 5.2
Page 103 and 104:
5. Service Delivery Model Table 5.3
Page 105 and 106:
5. Service Delivery Model I don’t
Page 107 and 108:
5. Service Delivery Model group is
Page 109 and 110:
5. Service Delivery Model Figure 5.
Page 111 and 112:
6. The Costs of headspace One of th
Page 113 and 114:
6. The Costs of headspace Table 6.1
Page 115 and 116:
6. The Costs of headspace In-scope
Page 117 and 118:
6. The Costs of headspace Revenue/
Page 119 and 120:
6. The Costs of headspace Other gov
Page 121 and 122:
7. Conclusion The evaluation of hea
Page 123 and 124:
7. Conclusion treatment); however,
Page 125 and 126:
7. Conclusion Overall the evaluatio
Page 127 and 128:
Appendix A Evaluation scope areas h
Page 129 and 130:
Appendix B The current model of hea
Page 131 and 132:
Appendix B and each SA3 in non-capi
Page 133 and 134:
Appendix B chapter 39 . To determin
Page 135 and 136:
Appendix B While there is an overal
Page 137 and 138: Appendix B Figure B3 describes the
Page 139 and 140: Appendix B Figure B5 Cost per young
Page 141 and 142: Appendix B The concept of access as
Page 143 and 144: Appendix B Figure B7 Heard of heads
Page 145 and 146: Figure B10 Proportion of headspace
Page 147 and 148: Appendix B Table B5 Females 18-25 y
Page 149 and 150: Appendix B following section. Figur
Page 151 and 152: Appendix B and less access to mains
Page 153 and 154: Appendix B Figure B17 Estimated pre
Page 155 and 156: Appendix B Figure B18 SA1s within 1
Page 157 and 158: Figure B20 Hypothetical allocation
Page 159 and 160: Appendix B The current model of cen
Page 161 and 162: Appendix B Current centre funding m
Page 163 and 164: Appendix B Evaluation of the curren
Page 165 and 166: Appendix B Level of funding availab
Page 167 and 168: Appendix B provide a method for ide
Page 169 and 170: Appendix B centres and Localities i
Page 171 and 172: Appendix B for greater flexibility
Page 173 and 174: Appendix B Additional consideration
Page 175 and 176: Appendix B of achieving more effici
Page 177 and 178: Appendix B • Used the simulation
Page 179 and 180: Appendix C information relating to
Page 181 and 182: Appendix C Regarding gender, in bot
Page 183 and 184: Appendix C Figure C5 Total number o
Page 185 and 186: Appendix C Table C1 Information Sum
Page 187: Appendix C Data Cleaning and Analys
Page 191 and 192: Appendix C Figure C8 Demographic ch
Page 193 and 194: Appendix C The KeySurvey software s
Page 195 and 196: Appendix C different client groups.
Page 197 and 198: Appendix D Round State Centre Name
Page 199 and 200: Appendix E Characteristics HS clien
Page 201 and 202: Table E4 Cell Sizes for Figure 3.4
Page 203 and 204: Appendix F Figure F1 Changes in K10
Page 205 and 206: Appendix F three SEIFA quintiles. A
Page 207 and 208: Appendix F Occasion of service inte
Page 209 and 210: Appendix F Table F1 Cell sizes for
Page 215 and 216: Appendix G Supplementary material f
Page 217 and 218: Appendix G Table G3 Round 3 (10 cen
Page 219 and 220: Appendix G Table G7 Round 7 (10 cen
Page 221 and 222: State (youth population) a headspac
Page 225 and 226: Appendix G Table G12 Round 11(16 ce
Page 227 and 228: Table G16 Additional youth populati
Page 229 and 230: Round Region State Additional youth
Page 235 and 236: Appendix G Figure G1 Percentage of
Page 237 and 238: Appendix G Figure G5 Percentage of
Page 239 and 240:
Appendix G Figure G9 Percentage of
Page 241 and 242:
Appendix G Figure G13 Percentage of
Page 243 and 244:
Appendix G Table G22 Australians 18
Page 245 and 246:
Appendix G Synthetic small area est
Page 247 and 248:
Appendix G Figure G16 Estimated pre
Page 249 and 250:
Page 251 and 252:
Page 253 and 254:
Appendix G Figure G22 SA1s within 1
Page 255 and 256:
Page 257 and 258:
Page 259 and 260:
Appendix G Table G25 Centres alloca
Page 261 and 262:
References Atkins, D.C., Bedics, J.
Page 263:
References Smith, E., Jones, T., Wa
show all

Is headspace making a difference to young people’s lives?

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?