pdf - Institute for Policy Research - Northwestern University

J. Reblando P. Reese
Quantitative Methods for Policy Research
MMost researchers and academics tend to stick with the research methods they
know best, learned mainly in graduate school—even though those methods
might not represent current best practices or the most appropriate method.
This is one reason why IPR Faculty Fellow Larry V. Hedges, with the support
of a group of distinguished interdisciplinary scholars, launched the Center for
Improving Methods for Quantitative Policy Research, or Q-Center, at the
Institute for Policy Research. Hedges, who is Board of Trustees Professor of
Statistics and Social Policy, co-directs the center with Thomas D. Cook, Joan
and Sarepta Harrison Chair in Ethics and Justice. Q-Center faculty work on:
• improving designs, analysis, and synthesis in policy research,
• designing better research methods for education,
• fostering a community of scholars, and
• developing new data sources and methods of data collection.
Larry Hedges,
Co-chair
8 Overview of Activities
Methodology and Research Designs
Economist Charles F. Manski continues
his line of work on the difficulties
of selecting the best policy with
limited knowledge of policy impacts.
Manski, Board of Trustees Professor in
Economics, finished his forthcoming
book Identification for Prediction and
Decision (Harvard University Press)
that exposits his new methodology for
analyzing empirical questions in the
social sciences. He recommends that
researchers first ask what can be learned
from data alone and then ask what can
be learned when data are combined with
credible weak assumptions. Inferences
predicated on weak assumptions, he
argues, can achieve wide consensus, while
ones that require strong assumptions
almost inevitably are subject to sharp
disagreements.
Thomas D. Cook, Joan and Sarepta
Harrison Chair in Ethics and Justice,
conducted a review of the history of
the regression discontinuity design
(RDD) in psychology, statistics, and
economics that will appear in the Journal
of Econometrics. Donald T. Campbell, who
invented the design in 1958, and a group
of Northwestern University colleagues,
including Cook, worked on RDD until
the early 1980s when the design fell into
disfavor. Cook speculates on why RDDs
held such a low profile until the mid-
1990s. Since then the design has widely
caught on, particularly among younger
econometricians and labor economists
in both the United States and Europe.
Cook suggests why this 50-year-old design,
rarely used until the beginning of this
century, has been reborn.
Cook and IPR graduate research
assistant Vivian Wong have published
a paper reviewing whether regressiondiscontinuity
studies reproduce the
results of randomized experiments
conducted on the same topic. They
enumerate the general conditions
necessary for a strong test of
correspondence in results when an
experiment is used to validate any nonexperimental
method. They identify three
past studies where regression discontinuity
and experimental results with overlapping
samples were explicitly contrasted. By
criteria of both effect sizes and statistical
significance patterns, they then show that
each study produced similar results. This
Thomas Cook,
Co-chair
Q-Center faculty
conduct research
relevant to the
center’s mission of
improving designs,
data collection,
analysis, and
synthesis in social
policy research.
www.northwestern.edu/ipr 43