Full report. - Social Research and Demonstration Corp

learn$ave Project: Final ReportAssistance (IA) recipients who received learn$ave-plusservices and the matched credits but were not part of theexperimental study.At the non-experimental sites, the target of 150participants, who received learn$ave-plus services andcredits, was attained for the most part. The main exceptionwas in the large rural area of Grey-Bruce where just101 participants could be enrolled (Table 3.1). This madefor a total of 1,001 participants at these sites, of whom466 were IA recipients. The number of IA recipients wasoriginally limited to 25 per cent of a site’s participants.However, Winnipeg received special dispensation toexceed the limit owing to the nature of its clientele, andGrey-Bruce reached its IA target of 25 per cent of 150participants before realizing it could not reach its totaltarget of 150.Survey dataBaseline and follow-up telephone surveys conductedby POLLARA Inc. (under contract with SRDC) were thesource of the outcome data used in the experimentalstudy for the estimation of impacts. 1 At the experimentalsites, shortly after meeting the eligibility criteria andbefore random assignment to one of the three researchgroups, applicants were surveyed to gather baselineinformation on personal and family characteristics.Participants were then re-contacted in follow-up surveysat 18, 40, and 54 months from the date of their lastinterview to gather outcome data in order to measureimpacts. Data from the final 54-month follow-up surveywere the source for impact estimates presented later inthis report, with comparisons to published results fromearlier 18- and 40-month surveys.A total of 2,269 participants responded to the 54-month survey, out of the original individuals randomlyassigned in the project (Table 3.2). This translates intoa response rate of 63.3 per cent. The 54-month surveyoccurred between September 2006 and September 2008,with an average survey interview length of about 34minutes. This is considerably lower than the 55-minuteaverage for the 40-month survey, for at least threereasons: first, only a 14-month period over which to askrespondents to recall their education activities, comparedto a 22-month period in the 40-month survey (40 less 18months); second, familiarity on the part of interviewersand interviewees with the routine and the questions;and third, a lack of contact module, which was used inprior surveys to update participants’ contact information.Owing to the project time constraints, the 102participants in the last three month-cohorts were calledin the 54-month survey (as with the 40-month survey) 3,1 Data sources used in other aspects of this study are described in Appendix C.2, and 1 month(s) early, respectively, as part of the fourthlast month-cohort in June 2008. This meant that therewas somewhat less time for these individuals than earliercohorts (1) to be contacted (though there were only 10who still were being called when the survey closed) and(2) to have saved and used their credits for education orsmall business.Between the 40- and 54-month surveys, survey responseactually rose as attrition appears to have levelledoff. One reason is that many of the participants who weregoing to “drop out” had already done so prior to the latersurveys. The plateauing of attrition was also due to twomeasures introduced by SRDC:• the introduction, after the first months of the 54-monthsurvey, of a $50 response incentive for all respondents. 2(In the 40-month survey, the response incentive had been$20 or $50 upon first refusal.); and• the use of the PMIS to obtain contact information on theprogram group participants to bolster survey response.The PMIS was felt to have more up-to-date contactinformation than the survey data. Indeed, this appearedto have worked somewhat, as the untraceable rate fellfrom the 40-month to the 54-month survey. 3As in all follow-up surveys, the 54-month surveyresponse rate for the control group was lower than itwas for the learn$ave-only and learn$ave-plus programgroups, with the gap widening somewhat between groupssince the 18-month survey (Table 3.2). This is not surprisinggiven that control group program members likelyfelt less engaged in the project and thus less inclinedto participate in the surveys. Most of the differences inresponse rates can be attributed to the higher incidenceof refusals and untraceable participants in the controlgroup than in the program groups, as the table indicates.The unbalanced attrition (difference in response ratesbetween program and control groups) did not negativelyaffect the socio-demographic comparability of the groupsto a significant extent and therefore the ability to measureimpacts by comparing the outcomes of the groups. Ifsignificant differences had appeared, then differences in2 As there was some fear that the increased incentive for everyone would mean an overalllower income respondent pool, comparisons were made between those responding beforeand after the incentive was introduced. However, there were few differences in baselinecharacteristics between these groups.3 There was some concern that, as the PMIS contained information on the program groupsbut not on the control group, this would create systematic bias between research groups.However, respondents found through PMIS contact information were found to be no differentfrom those who were contacted right away or through contact information found in regulardatabase searches.Social Research and Demonstration Corporation Chapter 3 | 27