OTIS Evaluation Guide (PDF) - California Department of Public Health

How will the sample be selected?
Census, simple random sampling, cluster sampling, stratified sampling, purposive sampling, and convenience
sampling are common sampling methods. Please see definitions alongside the page for more details about
these sampling methods. We’ll use an example that illustrates several of these sampling methods. Let’s say you
are going to use the Youth Tobacco Purchase Survey in order to determine STAKE Act compliance. After you
have assigned six communities to either an intervention or control group, you notice there are thousands of
stores selling tobacco products within each community. You realize that it is impossible to conduct the survey
for all the stores (a census) because you only have limited resource. Having a limited resource could also be
problematic if some stores were located far apart and would require extensive travel time.
Several options are available to select the stores for your sample:
• Random sample of stores (simple random sample ): First, you can randomly select a
specified number of stores in the intervention communities and the same number of stores in the control
communities. Random selection implies that every store in the intervention community and control
community has an equal opportunity to be selected to receive a compliance check. Using a random
selection will yield an ideal sample.
• Random cluster of communities (random cluster sample ): In another scenario, your
county territory is very large and you are a lone ranger without any available local coordinator, and
flying back and forth is out of the question due to your limited budget. In this case, you may want to
consider possibilities other than a simple random sample. Another option is to randomly select several
communities in the intervention regions and the same number of communities in the control regions and
then conduct compliance checks in the stores in these selected communities.
• Purposive sample of stores : A third option may be preferable when there are few resources
available. You purposively select stores from City A and City B because you know they have the best representation
of the county. The compliance checks would be conducted in these purposively selected stores.
• Convenience sample of stores : This next option is inferior to the other three options and should
only be used as a last resort. As opposed to randomly selecting stores in each region, community or
city, you handpick stores from the intervention region, and the same number of stores from the control
region. You should always try to get a random sample first, and use a convenience sample only if randomization
is indeed impractical. Sometimes a non-random sample is the only choice. For example, if
you plan to conduct a survey at a local health fair, you may need to get as many participants as you can.
• Stratified sample : This is an interesting alternative. If you have some previous knowledge that
the rate of the STAKE Act violation varies depending on the type of store, you may consider “stratifying”
all the stores in the county as either a small store (e.g., convenience store, mom-and-pop store, etc.) or
a large store (e.g., chain store). You can then take a random sample from the group of small stores and
large stores separately.
How many will be in the sample and why?
Addressing the issue of sample sizes is extremely important because even if you have a great program and a
high quality study design, a too-small sample size will weaken your evaluation considerably.
OTIS Evaluation Guide • March 2007
87