Asking Questions - The Definitive Guide To Questionnaire Design ...

QUESTIONNAIRES FROM START TO FINISH 319
viewers. It will be necessary to prepare instructions for these interviewers
and to train them on how the questionnaire should be
asked. In this process the trainer will often find ambiguities in the
questionnaire that must be corrected before training can continue.
The pilot test procedure is not identical to the main study. The
sampling is generally loose, and interviewers are given some flexibility.
If, however, the main study will be conducted with respondents
who might have trouble with some of the words or ideas in
the questionnaire, such respondents must be included in the pilot
sample. Also, the interviewers typically discuss the questionnaire
with the respondents after the pilot-testing interview is over, to
discover whether any of the questions were unclear or difficult
to answer. It is also very helpful for questionnaire writers or field
supervisors to observe some pilot-test interviews, since they may
find questions that are being misinterpreted by interviewers and
respondents. For complex studies it is always useful to meet with
interviewers after the pilot study to learn what problems they and
the respondents had. For more simple studies, this information can
be obtained in interviewer reports and from the comments written
on the questionnaire.
Pilot testing of mail and other self-administered questionnaires
must be conducted a little differently. The preferred procedure is
to mail or give the questionnaire to respondents, with no indication
that the questionnaire is not in its final version. After the
questionnaire is returned, telephone or face-to-face interviews are
conducted with some or all of the respondents, to determine
whether they had any difficulties in understanding or answering
the questions.
The process just described should not be confused with a field
trial of a preliminary version of the questionnaire. Although such
field tests can be desirable, they have different purposes and should
always follow the more informal review process just described. A
field trial will be desirable or necessary if there is substantial uncertainty
in what the response rate of the questionnaire is likely to be.