Asking the Right Questions, A Guide to Critical Thinking, 8th Ed

How Good Is the Evidence 11? 135
speech. If you have something bad to say about the U.S., say it, but do not
cheapen the flag with your actions. Many Americans died to keep that flag flying.
Those who want to support flag burning and other such despicable acts are outnumbered.
Last month, 75 people were surveyed in a restaurant in Dallas, Texas,
and were asked if they supported the unpatriotic desecration of the American flag
in an attempt to express some sort of anti-American idea. Ninety-three percent
responded that they were not in favor of desecration of the American flag.
Therefore, our national lawmakers should pass a law protecting the American
flag against such horrible actions.
Sample Responses
Passage 1
CONCLUSION: Children of alcoholics are more likely to become alcoholics than are child
of non-alcoholics.
REASON: More alcoholics than non-alcoholics reported a substantially higher rate of
having an alcoholic parent.
Note that the results presented are from one study without reference to how
typical these results are. We also do not know where this information was
published, so we can make no assessments regarding how rigorously the study
was reviewed before publication. However, we can ask some useful questions
about the study. The sample size is quite large, but its breadth is questionable.
Although multiple states were sampled, to what extent are the people in the
AA programs in these states typical of alcoholics across the nation? Also, how
do alcoholics in AA compare to alcoholics who have not sought help? Perhaps
the most important sampling problem was the lack of a random sample. While
the self-reported non-alcoholics were randomly selected in the three states,
the respondents in AA were selected on a voluntary basis. Do those who
volunteered to talk about their parents differ greatly from those who did not
volunteer? If there is a difference between the volunteers and non-volunteers,
then the sample is biased.
How accurate are the rating measurements? First, no definition for alcoholic
for those answering the survey is given beyond currently being in AA.
In addition, we are not told of any criteria given to the research participants
for rating parents as alcoholic. Thus we are uncertain of the accuracy of the
judgments about whether someone was an alcoholic. Also, problematic is the
fact that the selection of the supposed control group of non-alcoholics is based
on self-assessment. We know that there is a socially acceptable answer of not