Russel-Research-Method-in-Anthropology

286 Chapter 10
they sent out questionnaires to a national sample of 1,310 and got back 765,
or 58%. Not bad for a first pass, since you can generally expect about 25% to
30% from the first wave.
Unfortunately, for lack of time and money, Thurman et al. couldn’t follow
through with all the extra mailings. Of the 765 respondents, 237 said they
were nondrinkers. This left 525 eligible questionnaires for analysis.
Of the 525 respondents who said they were consumers of alcohol, 133
admitted driving while drunk in the past year. Those 133 respondents provided
data of intrinsic interest, but the 765 people who responded from the
nationally representative sample of 1,310 may be a biased sample on which to
base any generalizations. I say ‘‘may be’’ a biased sample because there is no
way to tell. And that’s the problem.
The bottom line: The last interview you get in any survey—whether you’re
sending out questionnaires, doing a phone survey, or contacting respondents
for face-to-face interviews—is always the most costly and it’s almost always
worth it. If you really care about representative data, you won’t think of all
the chasing around you have to do for the last interviews in a set as a nuisance
but as a necessary expense of data collection. And you’ll prepare for it in
advance by establishing a realistic budget of both time and money. (For recent
examples of mailed surveys using the TDM, see Gore-Felton et al. [2002] and
Filip et al. [2004].)
Pretesting and Learning from Mistakes
There is no way to emphasize sufficiently the importance of pretesting any
survey instrument. No matter how much you do to prepare a culturally appropriate
questionnaire, it is absolutely guaranteed that you will have forgotten
something important or that you will have poorly worded one or more vital
elements. These glitches can only be identified by pretesting.
If you are building a self-administered questionnaire, bring in at least 6 to
10 pretest respondents and sit with them as they fill out the entire instrument.
Encourage them to ask questions about each item. Your pretest respondents
will make you painfully aware of just how much you took for granted, no
matter how much ethnographic research you did or how many focus groups
you ran before making up a questionnaire.
For face-to-face interviews, do your pretesting under the conditions you
will experience when the survey is underway for real. If respondents are going
to come to your office, then pretest the instrument in your office. If you are
going to respondents’ homes, then go to their homes for the pretest.
Use the thinkaloud method (also called cognitive testing) on at least a few