planning a governance assessment - United Nations Development ...

Survey data and survey research is a relatively new field. Modern surveys were pioneered in the US and elsewhere
after World War II. As surveys became more common and computers more powerful, social scientists became interested in
conducting and designing surveys. Social scientists examined closely how question wording, question order and different
types of response sets influence results. They found that questionnaire design is critically important in terms of the quality
of results. These small, easily avoided mistakes can cause big problems with results and quality of data, especially in terms of
validity and reliability. It is wise to take advantage of the research on these issues when examining existing indicators, survey
questions or when developing new ones. The best advice is to have an outside survey research expert, with many years of
experience, examine the questionnaire in the draft phase of development.
Survey data comes in many forms. Original data comes from research you conduct yourself, and secondary survey data
comes from projects conducted by others. Data from surveys can be qualitative, quantitative or a combination of both.
Surveys are primarily used to determine the de facto governance situation. However, in some circumstances researchers
often use surveys to test the knowledge of respondents concerning de jure governance.
Types of interviewing
• Focus groups generate qualitative data and are good for developing questions and gaining a deeper understanding
of issues.
• Structured interviews use an identical instrument for each respondent. Interviewers are given explicit instructions.
This technique has a systematic approach that uses primarily structured questions with fixed response sets. Usually
very few open-ended questions or questions asking for detailed comments exist.
• In a semi-structured interview a written list of questions and topics that need to be covered in a particular order is
outlined. These questionnaires are often developed from informal, unstructured and focus group interviews. They
can include open-ended and/or more structured questions. This approach is ideal when working with elites, managers,
bureaucrats and other people who have limited time.
• Some projects combine elements by starting with a large structured quantitative study and then selecting a portion of
the sample for more in-depth, semi-structured qualitative interviewing on selected topics or indicators
Modes of survey data collection
Face-to-face data collection is likely the best option in most developing countries. It is also the most expensive and
time-consuming. It requires professional management of trained interviewers, the sample itself, and other aspects of the
study. In many countries, it is the only way to reach respondents and achieve a reasonable response rate. Data collected
using this method must be cleaned and entered twice to assure accuracy, thereby further increasing the cost of this mode
of data collection. One way to reduce costs without compromising too much on quality is to use university students who are
often interested in gaining research experience. Another way is to take advantage of NSOs who have trained staff and vast
experience in this type of data collection.
Mail surveys can work well only if the postal system is reliable. The questionnaire must be carefully designed for selfadministration,
and there should not be too many language issues. The designing and implementation of selfadministered
surveys is a well researched sub-field of survey research. With the right population, such as businesspeople or
other elites with a valid mailing address, mail surveys can work very well. The cost is usually quite reasonable. Plan on making
at least three attempts/mailings for each respondent to meet the “best practices” threshold for this mode. This method is not
recommended for surveys of typical citizens in most developing countries. It also misses the homeless and other vulnerable
groups who may not have a valid mailing address.
Planning a Governance Assessment: A Guide to Approaches, Costs and Benefits
17