social welfare research institute - Boston College

Reflections on Validity
The methodology of intensive interviews coupled with the study’s small sample
make it impossible for us to have captured a thorough portrait of the full population of
high-tech donors. Nevertheless, the number of interviews is sufficient for us to speak
with some assurance about the main lines of agent-animated philanthropy. Although we
did not conduct a representative random sample, our branching and direct email methods
of enlisting respondents did lead us across many regions of the country, and into the lives
of many different individuals. Even though the respondents who agreed to be
interviewed are likely to be more philanthropically active, and are not likely to be good
informants about those who are not so highly involved, the respondents were diverse
enough and spoke consistently enough for us to identify an array of common motifs.
In all interview case studies such as this, the issue of validity is not a statistical,
but an intellectual one. The question is not whether the sample represents the full range
of variation in targeted population, but whether what we learn is true for the sub-
population we actually interview. How far beyond the interview sample we can apply the
findings is a second question, one we address in the conclusion when we discuss the
implications of the study. As to the intellectual validity of the study, one helpful criterion
is that the more researchers hear the same set of dilemmas and strategies from a relatively
diverse group of respondents, the greater their assurance that they have discovered some
prevalent themes. A second working criterion is that when researchers begin to hear
those themes repeated in subsequent interviews, there is good reason to believe that they
have captured a good part of the story of a good number of people, and that additional
37