Background Methods Results Results (cont') - 2003 National ...

AL-Alabama, AK-Alaska, AZ- Ar i zona, AR-Ar kansas, CA-Calif or nia, CO-Color ado, CT -Connecticut,
DE-Delaware, DC-District of Columbia, FL-Florida, GA-Georgia, HI-Hawaii, ID-Idaho- IL-Illinois- IN- Indiana, IA-Iowa, KS-Kansas, KY-
Kentucky, LA-Louisiana, M E - M aine, M D- M ar yl and, M A- M assachusetts, Ml- Michigan, MN- Minnesota, MS- M ississippi, MO- M issour i,
MT - M ontana, NE- Nebr aska, NV- Nevada, NH- New Hampshir e, NJ- New Jersey, NM - New Mexico, NY- New Yor k, NC- Nor th Car olina, ND-
Nor th Dakota, OH- Ohio, OK- Okl ahoma, OR- Or egon, PA- P ennsylvania, RI- Rhode I sl and, SC- South Car olina, SD- South Dakota, T N-
Tennessee, T X- Texas, UT - Utah, VT- Ver mont, VA- Vir ginia, WA- Washington, WV- West Vir ginia, WI- Wisconsin, WY- Wyoming.
A State Level Evaluation Design To Assess
Program Effectiveness
Priti Bandi 1 , Patrick L. Remington 2 , D. Paul Moberg 2
American Cancer Society 1, University of Wisconsin
Comprehensive Cancer Center 2
•Acknowledgements
‣This project was funded by the Tobacco Prevention and Control Program, Division
of Public Health, Wisconsin Department of Health and Family Services.
‣The authors would like to thank David Ahrens, Barbara Hill and Vilma Cokkinides
for their contributions.
‣The authors would like to thank the American Cancer Society for their sponsorship
of this poster presentation.
•Contacts
‣Priti Bandi – priti.bandi@cancer.org
‣Patrick L. Remington – plreming@factsaff.wisc.edu
‣D. Paul Moberg - dpmoberg@wisc.edu
Background
• State level tobacco control program evaluations have traditionally employed state-tostate
or national comparisons, of tobacco use measures, in assessing program
effectiveness. This methodology, while accounting for secular trends, does not
account for complex factors affecting tobacco use at the state level.
• This research presents program evaluators an innovative design to conduct program
evaluations by comparing state level outcomes to those in unique groupings of “peer”
states.
• Wisconsin’s progress in reducing cigarette consumption, two years after program
implementation, is used to illustrate this methodology.
• Wisconsin’s Comprehensive Tobacco Control Program
‣ Wisconsin Tobacco Control Board established in 1999.
‣ Most Board instituted programs became operational in Summer 2001.
‣ Funding – Approx $21 million (Jan-Dec ’01) , Approx $ 13 million (Jan ’02-present)
‣ Seven primary goals were established, one of which included a 20 % reduction
capita cigarette consumption between 2000 and 2005.
Methods
• Cluster analysis
‣ Multivariate cluster analysis (Ward’s hierarchical method) was used to classify all
50 states and the D.C. into distinct homogenous clusters.
‣ State-level variables related to state-level cigarette consumption were included in
the cluster analysis. These variables were divided into quintiles (1-lowest, 5-
highest).
‣ Sociodemographic - State education level (% in state over age 25 with
bachelor’s degree) and state Hispanic ethnicity level (% of state population
that is Hispanic or Latino). 1
‣ Tobacco policy related - Inflation adjusted real price of cigarettes 2 and a
scale of state-level smoke-free air laws in 5 public places 3 (Range=0 to 15,
Mean=5.2, Cronbach’s alpha = 0.74).
‣ Validity of the cluster solution was established by a comparison of means using
univariate F ratios.
• Outcome measure - Per capita cigarette sales 2 were used to estimate cigarette
consumption.
‣ Fiscal year 2001 was established as baseline year of the program. Annual and
cumulative percent changes in Wisconsin between 2001-2003 were compared to
corresponding changes in its “peer” state cluster (defined as the state cluster that
contained Wisconsin) and other state clusters.
• Cluster analysis
‣ Cluster solution – Two, four and
five cluster solutions from the
hierarchical cluster analysis were
examined. Based on the
agglomeration coefficients, the
dendogram from the hierarchical
analysis, and the cluster profiles
on the variables included in the
analysis, a four cluster solution
yielded the most parsimonious
and explicable solution.
‣ Cluster validity – Significant
differences were observed across
clusters on the clustering
variables (internal), variables that
were not included in the analysis
(external) and, those variables
that have a theoretically based
relationship to the clustering
variables (predictive). These
results suggested validity of the
cluster solution. (Table 1)
Results
Cluster 1 Cluster 2 Cluster 3
(n=16) (n=13) (n=11) †
Cluster 4
(n=11)
Differences
across
clusters
p values
Internal validity
State cigarette price (cents) 156.4 167.3 185.3 199.2 p

‣ Cluster interpretations (Figure 3)
While the high and highmedium
consumption clusters
were similar on state cigarette
price and smoke-free air scale,
they were different on state
sociodemographic
characteristics, especially
Hispanic ethnicity level.
The low consumption and
low-medium consumption
clusters were similar on all the
variables except on Hispanic
ethnicity level, where the lowmedium
consumption cluster
was closer to the high
consumption cluster.
Results (cont’)
M ean of the clustering v a r ia b le (q uintile)
5.0
4.5
4.0
3.5
3.0
2.5
2.0
1.5
1.0
0.5
0.0
High consumption cluster
Low -medium consumption cluster
State Cigarette
Price
High-medium consumption cluster
Low consumption cluster
State smoke-free State education State Hispanic
scale
level
ethnicity
Figure 3. Cluster profiles for the 4 cluster solution
Results (cont’)
Progress in reducing cigarette consumption in Wisconsin
Table 2: Comparison of Wisconsin to State Groups-Cigarette Consumption* and Percent Change 2001-2003
% % %
Cigarette Cigarette Cigarette Change Change Change
Consumption Consumption Consumption 2001- 2002- 2001-
2001 2002 2003 ‘02 ‘03 ‘03
Mean 98.6 96 95.7 -2.3 -0.9 -3.5
High consumption cluster (n=16) Median 98.3 96.9 91.4 -2.5 -2.4 -4.4
High-medium consumption cluster Mean 77.3 74.5 75.4 -3.7 0.6 -3.1
(n=13) Median 78.6 77.9 74.7 -2.9 1 -3.3
Low-medium consumption cluster Mean 87.8 87 86.9 -0.8 -3.3 -4.2
† (n=10) Median 78.7 77.8 72 -1.7 -6.9 -5.8
Mean 53.6 53.5 47 0.3 -11.3 -10.8
Low consumption cluster (n=11) Median 50.2 51.1 47.5 1.1 -10.1 -13.3
Wisconsin 78.2 75.3 71 -3.7 -5.7 -9.2
* Packs per person,
Group 1- Alabama, Arkansas, Indiana, Iowa, Kentucky, Louisiana, Mississippi, Missouri, Montana, North Dakota, Ohio,
Oklahoma, South Carolina, South Dakota, Tennessee, West Virginia
Group 2 († Wisconsin excluded from this group: Peer state group) – Alaska, Delaware, Kansas, Maine, Maryland,
Michigan, Minnesota, Nebraska, New Hampshire, Vermont
Group 3 – Arizona, Colorado, Florida, Georgia, Idaho, Nevada, New Mexico, North Carolina, Oregon, Pennsylvania,
Texas, Virginia, Wyoming
Group 4 – California, Connecticut, District of Columbia, Hawaii, Illinois, Massachusetts, New Jersey, New York, Rhode
Island, Utah, Washington
Table 2: Comparison of Wisconsin to State Clusters-Cigarette Consumption and Percent Reduction 2001-2003
Results (cont’)
Results (cont’)
Cigarette consumption
95
90
85
80
75
70
65
60
United States Peer State Cluster Wisconsin
Most programs became operational
in Summer 2001
2000 2001 2002 2003
mean = 4.2%
median = 5.8%
3.8%
9.2%
% Decline in
Consumption
baseline
(2001)-2003
• Showing a greater percent decline in its cigarette consumption rate than 3
of the four derived clusters, Wisconsin performed favorably in its progress
towards reducing cigarette consumption in the first two years of its program.
• Only Cluster 4 showed a greater percent decline than Wisconsin during this
time period.
• Wisconsin performed considerably better than its own peer state group
(Cluster 2), between 2001 and 2003, with a cumulative percent decline of
9.2 percent compared to 4.2 percent for the peer state cluster. In the first
year of the program, between 2001 and 2002, cigarette consumption in
Wisconsin declined 3.7 percent while consumption in its peer state declined
0.8 percent. In the second year of the program, between 2002 and 2003,
cigarette consumption in Wisconsin declined 5.7 percent compared to a
decline of 3.3 percent in its peer state cluster.
• Wisconsin’s cumulative percent decline in the first two years of its program,
between 2001 and 2003, was 5.0 percentage points greater than the
decline in its peer state cluster.
Figure 4. Progress in reducing cigarette consumption, WI vs. “peer” state cluster & U.S.
Conclusions
• This research found that states clustered into four distinguishable groups on
characteristics related to cigarette consumption. Comparisons with these
clusters serves as a useful state level evaluation design.
• This methodology minimizes the effects of state level factors that would not
be accounted for in simple comparisons to the United States or other states.
• Apart from cigarette consumption, other measures of tobacco use, such as
adult and youth smoking prevalence, can also be compared to assess
program effectiveness.
• Greater declines in Wisconsin compared to its peer state cluster and most
state clusters suggest early effectiveness of Wisconsin’s tobacco control
program.
• Limitations: Cluster analysis is primarily an exploratory technique that is
dependent on the variables included. Other variables were investigated for
inclusion and the choice of included variables reflects a mixture of past
research 4,5 and model parsimony. Cluster validity could not be established
by replication because the 51 states represented the universe of cases.
However, significant differences on the included and external variables
indicated validity. Also, there may be some states in each cluster that do not
conform to the description of the “average” state in that cluster. In that
respect, these derived clusters should be interpreted as separate entities
References
1. U.S. Census Bureau. Census 2000 Summary File 3-[United States]
American FactFinder. Available at: http://factfinder.census.gov Accessed
July 10, 2004.
2. Orzechowski W, Walker RC. The Tax Burden on Tobacco: Historical
Compilation. Volume 38, 2003. Arlington VA: Orzechowski and Walker;
2004.
3. ImpacTeen Project. ImpacTeen Tobacco Control Research Team. Roswell
Park Cancer Institute, Department of Health Behavior; Buffalo, New York.
Tobacco Control Policy and Prevalence Data: 1991-2001. Available at:
http://www.impacteen.org/tobaccodata.htm. Accessed: August 17, 2004.
4. Stillman FA, Hartman AM, Graubard BI, Gilpin EA, Murray DM, Gibson JT.
Evaluation of the American Stop Smoking Intervention Study (ASSIST): a
report of outcomes. J Natl Cancer Inst. 2003 Nov 19;95(22):1681-91.
5. Gilpin EA, Stillman FA, Hartman AM, Gibson JT, Pierce JP. Index for US
state tobacco control initial outcomes. Am J Epidemiol. 2000 Oct
15;152(8):727-38.
2