Primary Education Survey Evaluation Report Somalia - Somali - JNA

PES Evaluation Report, 2008
Study Components
The principal source of data will be interviews with respective stakeholders. Respondents
will include MoE officials at central, regional and district levels, NGO officials, head
teachers, teachers, survey enumerators, supervisors, UNICEF/UNESCO staff at Nairobi
and zonal level as well as local and international NGOs. The methods followed would be
largely qualitative and include key informant semi-structured interviews, community level
interviews and focus group discussions. Questionnaires for guiding the interviews will be
developed for review in the inception phase. (PES Evaluation TOR 2008)
The task of this methodology section is to coordinate these process components with the 7 areas identified
above. The 7 areas are further clarified and designed in Appendix 4 by the use of 40 research questions
mapped onto the 7 areas. A research matrix was developed that linked relevant data sources
(stakeholders) with the research questions. This appendix is referred to as the Data Collection Planner.
This matrix was used to ensure that each research question had an associated data source to inform it, to
identify people who would reasonably be expected to have relevant knowledge and experience to inform
each research question, and to guide the development of the interview guides that would assist the
researcher in generating the conversation in the semi structured interview environments. These processes
were validated by consultation with a range of stakeholders who have personal and detailed knowledge of
the context of the research.
The Data Collection planner was used to develop seven Interview guides. These guides targeted UNICEF
officers, Directors General of Education, planning officers in each Ministry, Regional and District
Education Officers, School and community leaders, Survey team members, and NGOs/international
organisations. In addition, questions that could be informed by the desk review were identified.
Appendix 5 indicates the set of interviews and focus groups conducted in the data collection process.
The semi-structured interviews with individuals and focus groups were recorded and salient information
mapped onto research questions. These data were synthesized and interpreted by the consultant prior to
drawing conclusions and making recommendations.
To enhance the validity of the study a number of strategies were used that were drawn from the
qualitative research literature (LeCompte and Preissle, 1993). These involve enhancing the
“trustworthiness” of the research. The strategies included triangulation, building a comfortable non
threatening relationship with interviewees, creating an audit trail of data using a digital recorder in
interviews, seeking permission of interviewee to record the interview with an offer to turn the recorder off
if requested, seeking specific examples in support of claims being made, asking to see documentary
evidence of claims and comments, and providing a summary of conversations by listing major points and
seeking concurrence from interviewees.
A few further words on triangulation are warranted. This is a process where the researcher tries to get
multiple sources of evidence to assist in making conclusions in relation to the study. To this end, after
studying the Data Collection Planner (Appendix 4) it can be seen that each research question is to be
investigated with several stakeholders. Additionally, when claims are made attempts will be made to
triangulate the claim with documentary, or physical evidence. In general, matters of fact should be agreed
upon by all data sources. Some variability will be expected in views, or attitudes of a more subjective
nature. Often it can be expected that this variability is a function of varying positions in the system. For
example the Director General might see data as a valuable help to decision making and resource
allocation, whereas teachers might see the collection of the same data as an imposition that reduces the
time they have for lesson preparation.
There were a number of threats and limitations to the study that should be enumerated. During the period
of the study (July/August) schools were closed for the summer break and hence were not available for
observation in a natural setting. Attempts were made to contact some local principals and school
representatives for interview.
12