Kenya OVC-CT Programme Operational and Impact Evaluation

1.2.2 Evaluation designOver the course of the evaluation, three main activities will be undertaken: 51. A quantitative survey of households and communities2. Qualitative data collection3. A costing studyThese activities will use a number of instruments:• Household survey (a panel survey, with baseline + follow-up);• Community quantitative survey (baseline + follow-up);• Qualitative focus groups;• Qualitative in-depth interviews;• Programme financial documentation review.The core of the evaluation is to assess the impact of the programme on the recipients bycomparing them with a group of controls – similar households and children who do not benefit fromthe programme. Both groups are interviewed before the programme begins and after it has beenoperating for two years. The impact is assessed by comparing changes in the welfare of recipients,who should have improved as a consequence of the programme, to any changes in the controlhouseholds. The information on the control households is used to allow for any other changes thatmay be happening in the population in general and have nothing to do with the programme. Theevaluation will also assess the impact of imposing conditions on the recipients (on schoolattendance, etc) compared to a cash transfer alone. In the evaluation, the selection of locations tobenefit from the programme with conditions, the programme without conditions, or no programme(controls) was done randomly. This strengthens the robustness of the findings.The sample for the quantitative survey therefore consists of four groups. 6 They are:• A – households with OVCs in the programme areas, selected for inclusion in theprogramme. These are divided into two groups – those in areas with conditions vs thosewhere there are no conditions.• B – households with OVCs in control areas that meet programme criteria and should, intheory, have been selected by virtue of meeting the definitions of high need as defined bythe programme, if the programme operated there.• C – households with OVCs in programme areas, but not selected for inclusion in theprogramme.• D – households with OVCs in control areas that do not meet programme criteria and wouldnot (in theory) have been selected if the programme operated there.5 A series of organisational reviews were also planned and the first was implemented. No further reviews willbe undertaken, however, since the rest were cancelled in order to focus resources on the quantitativesurvey.6 It had initially been proposed to include a sample of non-OVC households to provide information on thecharacteristics of all households in the study population.4