Strengthening the Empirical Base of Operations Management
Strengthening the Empirical Base of Operations Management
Strengthening the Empirical Base of Operations Management
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
MANUFACTURING & SERVICE<br />
OPERATIONS MANAGEMENT<br />
Vol. 9, No. 4, Fall 2007, pp. 368–382<br />
issn 1523-4614 � eissn 1526-5498 � 07 � 0904 � 0368<br />
<strong>Streng<strong>the</strong>ning</strong> <strong>the</strong> <strong>Empirical</strong> <strong>Base</strong> <strong>of</strong><br />
<strong>Operations</strong> <strong>Management</strong><br />
Marshall Fisher<br />
<strong>Operations</strong> and Information <strong>Management</strong> Department, The Wharton School, University <strong>of</strong> Pennsylvania,<br />
Philadelphia, Pennsylvania 19104, fisher@wharton.upenn.edu<br />
informs ®<br />
doi 10.1287/msom.1070.0168<br />
© 2007 INFORMS<br />
Isuggest that <strong>the</strong> prospering fields <strong>of</strong> physics, medicine, and finance illustrate <strong>the</strong> value <strong>of</strong> a strong empirical<br />
dimension to research that is well integrated with <strong>the</strong>oretical research. I use empirical research in <strong>the</strong>se fields<br />
to formulate a framework for classifying empirical research and illustrate that framework with a few selected<br />
examples in operations management. I <strong>of</strong>fer some advice on data sources and approaches to conducting empirical<br />
research and suggest ways streng<strong>the</strong>ning empirical research in operations management. This is obviously<br />
a partial treatment <strong>of</strong> a large subject and represents my personal point <strong>of</strong> view. This paper should encourage<br />
comments by o<strong>the</strong>rs to fur<strong>the</strong>r develop <strong>the</strong> topic and to <strong>of</strong>fer alternative points <strong>of</strong> view.<br />
Key words: empirical research; econometrics; experiments; case research; data sources<br />
History: Received: December24, 2005; accepted: February 21, 2007.<br />
1. Introduction<br />
The field <strong>of</strong> operations management has accomplished<br />
much <strong>of</strong> which we can be proud, but like all healthy<br />
fields, we should constantly strive for improvement.<br />
One way to improve is to learn from role models.<br />
Please join me in a bit <strong>of</strong> introspection. What academic<br />
fields do you think are prospering and would be useful<br />
role models for operations management?<br />
My list would include academic medicine, physics,<br />
and finance, because all three have blended deep intellectual<br />
content with pr<strong>of</strong>ound impact on <strong>the</strong> world.<br />
These fields have been guided by big questions that<br />
have led to big ideas such as <strong>the</strong> germ <strong>the</strong>ory <strong>of</strong> disease,<br />
<strong>the</strong> <strong>the</strong>ory <strong>of</strong> relativity, <strong>the</strong> Manhattan Project<br />
(however you feel about <strong>the</strong> result, this project, which<br />
created <strong>the</strong> atomic bomb during World War II, was<br />
a remarkably ambitious endeavor), <strong>the</strong> idea that in a<br />
perfect capital market, <strong>the</strong> price history <strong>of</strong> a stock is<br />
<strong>of</strong> no value in predicting future prices, and <strong>the</strong> analytic<br />
approach to investing that has emerged in <strong>the</strong><br />
last three decades.<br />
By contrast, operations management has had big<br />
ideas such as <strong>the</strong> industrial revolution, mass production,<br />
<strong>the</strong> assembly line, <strong>the</strong> Toyota Production System,<br />
and statistical process control. Yet <strong>the</strong>se ideas have<br />
368<br />
not come from academia. 1 We have been late to <strong>the</strong><br />
game, focusing on developing a deeperma<strong>the</strong>matical<br />
foundation and understanding <strong>of</strong> <strong>the</strong>se concepts<br />
once <strong>the</strong>y had been identified. Although this is doubtless<br />
a contribution, it has tended to position our field<br />
as proving <strong>the</strong>se were good ideas decades after <strong>the</strong>y<br />
have been well accepted in <strong>the</strong> market place as great<br />
ideas.<br />
Why has this happened, and what can we learn<br />
from o<strong>the</strong>r fields to change it? I believe we are at risk<br />
<strong>of</strong> falling victim to <strong>the</strong> malaise von Neumann (1956,<br />
p. 2063) warned <strong>of</strong>:<br />
Ma<strong>the</strong>matical ideas originate in empirics ��� � But once<br />
<strong>the</strong>y are so conceived, <strong>the</strong> subject begins to live a<br />
peculiarlife <strong>of</strong> its own and is bettercompared to a<br />
creative one, governed by almost entirely aes<strong>the</strong>tical<br />
motivations ��� � As a ma<strong>the</strong>matical discipline travels<br />
1 Some might argue that this is overly harsh. The fa<strong>the</strong>rs <strong>of</strong> statistical<br />
process control were Walter Shewhart and W. Edwards Deming.<br />
Both had PhDs (although Shewhart’s was in physics), Shewhart<br />
worked at Bell Labs doing academic type research, and Deming<br />
was a pr<strong>of</strong>essor during part <strong>of</strong> his career in <strong>the</strong> business schools <strong>of</strong><br />
New York University and Columbia. A recent big development in<br />
operations management, large-scale s<strong>of</strong>tware systems for inventory<br />
management and factory scheduling, draws heavily on voluminous<br />
<strong>the</strong>ory in inventory control and factory scheduling published in <strong>the</strong><br />
operations research literature.
Fisher: <strong>Streng<strong>the</strong>ning</strong> <strong>the</strong> <strong>Empirical</strong> <strong>Base</strong> <strong>of</strong> <strong>Operations</strong> <strong>Management</strong><br />
Manufacturing & Service <strong>Operations</strong> <strong>Management</strong> 9(4), pp. 368–382, © 2007 INFORMS 369<br />
far from its empirical source, ���<strong>the</strong>re is a grave dangerthat<br />
<strong>the</strong> subject will develop along <strong>the</strong> line <strong>of</strong> least<br />
resistance, and that <strong>the</strong> stream, so far from its source,<br />
will separate into a multitude <strong>of</strong> insignificant branches.<br />
���whenever this stage is reached, <strong>the</strong> only remedy<br />
seems to me to be <strong>the</strong> rejuvenating return to <strong>the</strong> source:<br />
<strong>the</strong> reinjection <strong>of</strong> more or less directly empirical ideas.<br />
When I read this quote, I see what I fearourfield<br />
might become in a decade ortwo if we continue to<br />
move along our current trajectory. Certainly “<strong>the</strong> line<br />
<strong>of</strong> least resistance” in <strong>the</strong> journal review process leads<br />
to a stream <strong>of</strong> incremental, unobjectionable papers<br />
giving rigorous answers to narrow questions. If excellent<br />
research is an important question well answered,<br />
I believe we have been betterat answering than at<br />
posing important and interesting questions.<br />
I am certainly not <strong>the</strong> first to make this observation.<br />
Bertrand and Fransoo (2002, 2006) and Keys (1991)<br />
are among those who have argued for streng<strong>the</strong>ning<br />
our empirical dimension. The following excerpt from<br />
Bertrand and Fransoo (2006) strikes an eerie parallel<br />
with von Neumann’s 1956 warning:<br />
Initially, ���operational research was oriented ���towards<br />
solving real-life problems ��� � Especially in <strong>the</strong><br />
USA, a strong academic research line in OR emerged<br />
in <strong>the</strong> 1960s, working on more idealized problems ��� �<br />
However, much <strong>of</strong> this research lost its empirical foundations.<br />
Research methods have been primarily developed<br />
for ���<strong>the</strong>oretical research lines, leaving <strong>the</strong> more<br />
empirically oriented research lines for more than 30<br />
years in <strong>the</strong> blue ��� (p. 241).<br />
I would agree with von Neuman; <strong>the</strong> way to avoid<br />
<strong>the</strong> risk <strong>of</strong> separating “into a multitude <strong>of</strong> insignificant<br />
branches” is to have a healthy injection <strong>of</strong> empirics.<br />
I believe medicine, physics, and finance have<br />
prospered because <strong>the</strong>y have, each in <strong>the</strong>ir own way,<br />
figured out how to integrate empirics with <strong>the</strong>ory.<br />
Medicine has evolved a variety <strong>of</strong> refined protocols<br />
for conducting and evaluating empirical research,<br />
including clinical trials fordrugs and o<strong>the</strong>rnew treatments<br />
and epidemiological studies that mine population<br />
data fordisease correlates. Medicine also takes<br />
a broad view <strong>of</strong> empirical research that includes less<br />
structured contributions; for example, <strong>the</strong> New England<br />
Journal <strong>of</strong> Medicine publishes descriptions <strong>of</strong> interesting<br />
cases. In recognition <strong>of</strong> <strong>the</strong> importance <strong>of</strong> <strong>the</strong>ir<br />
three missions—research, teaching, and patient care—<br />
medical schools now have three distinct types <strong>of</strong> faculty:<br />
PhDs doing basic lab research; MDs, who teach,<br />
do basic lab research, and see patients; and clinician/<br />
educators who teach and see patients. The integration<br />
<strong>of</strong> <strong>the</strong>ory and practice is greatly facilitated because<br />
medical schools are both conducting lab research and<br />
seeing patients in close proximity. They have coined<br />
<strong>the</strong> phrase “from bench to bedside” to connote <strong>the</strong>ir<br />
ideal model. What a medical faculty membersees in<br />
clinical practice motivates his orherlab research, <strong>the</strong><br />
fruits <strong>of</strong> which are <strong>the</strong>n made available to patients<br />
via <strong>the</strong> clinical practice. In fact, medicine has created<br />
a name foractivities that bridge <strong>the</strong> lab and clinical<br />
practice: translational research.<br />
Theory and empirics have also been synergistic in<br />
physics and finance. Things observed empirically, be<br />
<strong>the</strong>y new particles in an accelerator or a tendency<br />
for stock prices to rise in January, provide interesting<br />
phenomena to be explained by new <strong>the</strong>ories, and<br />
that <strong>the</strong>orizing produces new conjectures to be tested<br />
in <strong>the</strong> “lab.” Examples <strong>of</strong> <strong>the</strong>ory guiding empirics<br />
would include conjectured new particles to be discovered<br />
in an accelerator and <strong>the</strong> perfect market <strong>the</strong>ory,<br />
which asserts that one can’t earn supernormal pr<strong>of</strong>its<br />
in <strong>the</strong> stock market from an analysis <strong>of</strong> a stock’s price<br />
history.<br />
A taxonomy <strong>of</strong> <strong>the</strong> types <strong>of</strong> empirical research<br />
will be helpful in exploring how we might learn<br />
from medicine, physics, and finance to create something<br />
similarforoperations management. <strong>Empirical</strong><br />
research can be grouped by a number <strong>of</strong> attributes;<br />
two that make sense are how structured and formal<br />
<strong>the</strong> interaction with <strong>the</strong> world is and whe<strong>the</strong>r <strong>the</strong><br />
goal <strong>of</strong> <strong>the</strong> research is to describe <strong>the</strong> world or to<br />
determine a recommended course <strong>of</strong> action based on<br />
empirical observations. This taxonomy can be represented<br />
by <strong>the</strong> 2 × 2 matrix shown in Figure 1.<br />
To help in understanding <strong>the</strong> matrix, Figure 2 has<br />
various types <strong>of</strong> empirical research in medicine placed<br />
into <strong>the</strong> cells where <strong>the</strong>y seem to best fit. A clinical<br />
trial to gain FDA approval for a new drug is a<br />
highly structured and defined process, and I placed it<br />
in <strong>the</strong> prescriptive box because <strong>the</strong> goal <strong>of</strong> <strong>the</strong> trial is<br />
to make a decision on whe<strong>the</strong>rto introduce <strong>the</strong> drug.<br />
At <strong>the</strong> o<strong>the</strong>r extreme, medical journals will report<br />
accounts by physicians <strong>of</strong> interesting cases <strong>the</strong>y have<br />
seen. While <strong>the</strong>re probably is an agreed format for<br />
reporting cases, I placed this in <strong>the</strong> lower-right box<br />
because <strong>the</strong> collection <strong>of</strong> data seems less structured
Fisher: <strong>Streng<strong>the</strong>ning</strong> <strong>the</strong> <strong>Empirical</strong> <strong>Base</strong> <strong>of</strong> <strong>Operations</strong> <strong>Management</strong><br />
370 Manufacturing & Service <strong>Operations</strong> <strong>Management</strong> 9(4), pp. 368–382, © 2007 INFORMS<br />
Figure 1 A Taxonomy <strong>of</strong> <strong>Empirical</strong> Research<br />
Interaction with <strong>the</strong> world<br />
Highly<br />
structured:<br />
Data and<br />
algorithms<br />
Less structured:<br />
Interviews and<br />
observations<br />
Prescriptive<br />
Goal <strong>of</strong> <strong>the</strong> research<br />
Descriptive<br />
than a clinical trial and <strong>the</strong>re is no immediate action<br />
prescribed by <strong>the</strong> observations <strong>of</strong> a single patient.<br />
Lab research on mice and epidemiological studies<br />
seem more structured to me, but aimed at a general<br />
understanding <strong>of</strong> a disease. It is hoped that this<br />
general understanding might eventually lead to a recommended<br />
treatment, but that is not <strong>the</strong> immediate<br />
goal. I had to think hard to come up with an entrant<br />
for <strong>the</strong> lower-left box but concluded that critical care<br />
paths qualify. These are detailed descriptions <strong>of</strong> <strong>the</strong><br />
exact procedure to be followed in a surgical procedure,<br />
including pre- and post-surgical steps. (See<br />
Wheelright and Weber1995 fora more extensive discussion<br />
<strong>of</strong> critical care paths.) As such, critical care<br />
paths are clearly prescriptive, but in <strong>the</strong> examples<br />
I have seen, <strong>the</strong> care path is determined largely by<br />
interviewing doctors and o<strong>the</strong>r health care practitioners<br />
regarding <strong>the</strong>ir opinion <strong>of</strong> best practice. Health<br />
outcomes research attempts to discover best treatment<br />
options by looking at a large sample <strong>of</strong> patients who<br />
Figure 2 <strong>Empirical</strong> Research in Medicine<br />
Interaction with <strong>the</strong> world<br />
Highly<br />
structured:<br />
Data and<br />
algorithms<br />
Less structured:<br />
Interviews and<br />
observations<br />
Goal <strong>of</strong> <strong>the</strong> research<br />
Prescriptive Descriptive<br />
Clinical trial for a<br />
new drug<br />
Critical care path<br />
Laboratory research<br />
on mice<br />
Epidemiological studies<br />
that mine population<br />
data for disease<br />
correlates<br />
Observation <strong>of</strong><br />
interesting cases<br />
Figure 3 <strong>Empirical</strong> Research in <strong>Operations</strong> <strong>Management</strong><br />
Interaction with <strong>the</strong> world<br />
Highly<br />
structured:<br />
Data and<br />
algorithms<br />
Goal <strong>of</strong> <strong>the</strong> research<br />
Prescriptive Descriptive<br />
Engineering<br />
S<strong>of</strong>tware implementation<br />
<strong>of</strong> algorithm deployed in<br />
a company and run daily<br />
Principles<br />
Less structured:<br />
Ohno invents Toyota<br />
Interviews and Production System,<br />
observations inspired by <strong>the</strong> principles<br />
<strong>of</strong> U.S. supermarkets<br />
<strong>Operations</strong> management<br />
econometrics<br />
Statistical analysis <strong>of</strong><br />
large data sets to<br />
discover drivers <strong>of</strong><br />
success in operations<br />
Case studies<br />
Interview and observe<br />
managers<br />
Research cases<br />
received different treatments for <strong>the</strong> same disease and<br />
identifying <strong>the</strong> treatment variant that correlates with<br />
<strong>the</strong> best results. To <strong>the</strong> extent that critical care paths in<br />
<strong>the</strong> future will be guided by outcomes research, <strong>the</strong>y<br />
would move toward <strong>the</strong> upper left box.<br />
Now we can think about how prior empirical<br />
research in operations management fits into this<br />
matrix. Figure 3 shows <strong>the</strong> matrix labeled with types<br />
<strong>of</strong> empirical research in operations management and<br />
with one ormore examples <strong>of</strong> each type. As one<br />
can infer from <strong>the</strong> array <strong>of</strong> activities depicted in this<br />
matrix, my own definition <strong>of</strong> empirical research is<br />
broad and includes any effort to ga<strong>the</strong>r and report<br />
information about real operations that is accurate, is<br />
intellectually deep, raises interesting research questions,<br />
and contains enough correct analysis to at least<br />
partially answer those questions. If you object that<br />
this definition <strong>of</strong> empirical research is overly broad<br />
and that only what I have called operations management<br />
econometrics deserves to be called research, feel<br />
free to think <strong>of</strong> this matrix <strong>of</strong> activities as field work. I<br />
would certainly acknowledge that some entries in <strong>the</strong><br />
matrix, such as interviews with managers, may not<br />
be a stand-alone publishable product. However, as<br />
described in §8, such activities can be extremely valuable<br />
as part <strong>of</strong> a broader program <strong>of</strong> activity involving<br />
o<strong>the</strong>rcells <strong>of</strong> <strong>the</strong> matrix.<br />
Including case studies and <strong>the</strong> implementation <strong>of</strong><br />
an algorithm in a single company in <strong>the</strong> matrix raises<br />
<strong>the</strong> question <strong>of</strong> whe<strong>the</strong>r research based on a single<br />
observation is valid. Certainly research based on a<br />
large number <strong>of</strong> observations is to be encouraged, but<br />
<strong>the</strong>re is also much to be learned from deep and exten-
Fisher: <strong>Streng<strong>the</strong>ning</strong> <strong>the</strong> <strong>Empirical</strong> <strong>Base</strong> <strong>of</strong> <strong>Operations</strong> <strong>Management</strong><br />
Manufacturing & Service <strong>Operations</strong> <strong>Management</strong> 9(4), pp. 368–382, © 2007 INFORMS 371<br />
sive analysis <strong>of</strong> a single orlimited number<strong>of</strong> observations.<br />
Eisenhardt (1989), Voss et al. (2002), Voss<br />
(2006), and Yin (1994) contain extensive discussions <strong>of</strong><br />
why and how to conduct effective case research. Voss<br />
(2006) in particular directly considers <strong>the</strong> question <strong>of</strong><br />
whe<strong>the</strong>r research based on a single observation makes<br />
sense.<br />
The next foursections will describe some examples<br />
<strong>of</strong> each <strong>of</strong> <strong>the</strong> four types <strong>of</strong> research. I want to emphasize<br />
that what follows is not intended to be a comprehensive<br />
survey <strong>of</strong> empirical research in operations<br />
management, but a few illustrative examples. Clearly,<br />
a great deal <strong>of</strong> o<strong>the</strong>r excellent work is omitted here<br />
because <strong>of</strong> space constraints. The reader will detect,<br />
and I hope forgive, a Wharton bias in this survey; it is<br />
not that this research is necessarily more meritorious,<br />
but that it was more familiar to me.<br />
2. Engineering<br />
Engineers in industry and academia alike design and<br />
build things, <strong>the</strong>n test <strong>the</strong>ircreations, first in <strong>the</strong><br />
lab and eventually in <strong>the</strong> field. In most engineering<br />
disciplines what gets built is a piece <strong>of</strong> hardware,<br />
but in our field <strong>the</strong> analog to hardware is usually<br />
<strong>the</strong> s<strong>of</strong>tware implementation <strong>of</strong> an algorithm. Since<br />
<strong>the</strong> earliest days <strong>of</strong> operations research and management<br />
science, a core paradigm has been to model a<br />
real-world problem, devise an algorithm to analyze<br />
<strong>the</strong> model, create a s<strong>of</strong>tware implementation <strong>of</strong> that<br />
algorithm, and <strong>the</strong>n observe whe<strong>the</strong>r <strong>the</strong> algorithm<br />
improves <strong>the</strong> performance <strong>of</strong> <strong>the</strong> function that was<br />
modeled.<br />
Using company data to estimate <strong>the</strong> parameters <strong>of</strong><br />
a model is a form <strong>of</strong> empiricism. Moreover, while it<br />
might seem that deploying an algorithm is <strong>the</strong> end,<br />
ra<strong>the</strong>r than <strong>the</strong> beginning, <strong>of</strong> research, I have found<br />
that so much is learned during <strong>the</strong> implementation<br />
process that this itself constitutes a type <strong>of</strong> empirical<br />
research. During implementation you are forced<br />
to verify and refine <strong>the</strong> details <strong>of</strong> your model, so you<br />
evolve a very precise definition <strong>of</strong> how a particular<br />
operations function works. You also discover properties<br />
<strong>of</strong> real data that influence algorithm design.<br />
Forexample, <strong>the</strong> application <strong>of</strong> <strong>the</strong> simplex method<br />
to real linear programming problems revealed that<br />
real problems have very sparse constraint matrices, a<br />
property that could be exploited in developing a more<br />
efficient algorithm. Finally, putting an algorithm into<br />
<strong>the</strong> hands <strong>of</strong> users teaches you how <strong>the</strong>y think about<br />
problem solving. For example, most users want to see<br />
more than <strong>the</strong> final solution for a given data set. They<br />
want to see <strong>the</strong> logic path that leads from <strong>the</strong> input<br />
data to <strong>the</strong> final solution, and <strong>the</strong>y want a sensitivity<br />
analysis that tells <strong>the</strong>m whe<strong>the</strong>rit is worthwhile<br />
for<strong>the</strong>m to take an action that might relax one <strong>of</strong> <strong>the</strong><br />
constraints <strong>of</strong> <strong>the</strong> problem.<br />
Vehicle routing is one context where much has been<br />
learned from <strong>the</strong> deployment <strong>of</strong> algorithms. Vehicle<br />
routing focuses on <strong>the</strong> efficient use <strong>of</strong> a fleet <strong>of</strong> vehicles<br />
that must make a number<strong>of</strong> stops to pick up<br />
and/ordeliverpassengers orproducts. The problem<br />
requires one to specify which deliveries or pickups<br />
should be accomplished by each vehicle and in what<br />
order so as to minimize total cost subject to a variety<br />
<strong>of</strong> constraints such as vehicle capacity and delivery<br />
time restrictions. Fisher (1995) surveys research in<br />
vehicle routing and describes a number <strong>of</strong> successful<br />
applications <strong>of</strong> vehicle-routing algorithms. Bartholdi<br />
et al. (1983) is one <strong>of</strong> my favorite vehicle-routing<br />
applications because <strong>of</strong> <strong>the</strong> novel algorithm used and<br />
<strong>the</strong> social contribution <strong>of</strong> its application. The authors<br />
describe <strong>the</strong> application <strong>of</strong> a routing system based<br />
on space-filling curves that <strong>the</strong>y developed for <strong>the</strong><br />
Atlanta, Georgia, branch <strong>of</strong> Meals-on-Wheels, which<br />
delivers hundreds <strong>of</strong> meals daily to those who cannot<br />
shop for<strong>the</strong>mselves.<br />
A space-filling curve is a mapping between a lower<br />
and a higherdimensional space. A space-filling curve<br />
from R 2 to R 1 induces a sequencing <strong>of</strong> all points in<br />
a plane and hence provides a heuristic for <strong>the</strong> planer<br />
traveling salesman problem. The authors use a particular<br />
space-filling curve to sequence all required deliveries<br />
for a given day. They <strong>the</strong>n assign deliveries in<br />
orderto a vehicle until fur<strong>the</strong>rassignment would<br />
exceed vehicle capacity. This process is repeated with<br />
successive vehicles until all deliveries have been<br />
assigned to a vehicles. The resulting algorithm is so<br />
simple that it was implemented on two Rolodex card<br />
files and shown to have significant positive benefit.<br />
As <strong>the</strong> result <strong>of</strong> decades <strong>of</strong> similar research in creating<br />
and applying vehicle-routing algorithms to particular<br />
real problems, we now have a rich understanding
Fisher: <strong>Streng<strong>the</strong>ning</strong> <strong>the</strong> <strong>Empirical</strong> <strong>Base</strong> <strong>of</strong> <strong>Operations</strong> <strong>Management</strong><br />
372 Manufacturing & Service <strong>Operations</strong> <strong>Management</strong> 9(4), pp. 368–382, © 2007 INFORMS<br />
<strong>of</strong> <strong>the</strong> many nuances <strong>of</strong> vehicle routing that arise in<br />
practice, a taxonomy <strong>of</strong> <strong>the</strong> various versions <strong>of</strong> real<br />
vehicle routing problems, and a knowledge <strong>of</strong> vehiclerouting<br />
data structures. For example, Fisher (1994)<br />
observes that <strong>the</strong> delivery points in real vehiclerouting<br />
problems are highly clustered and discusses<br />
how to exploit this in an optimization algorithm. If<br />
<strong>the</strong> essence <strong>of</strong> empirical research is ga<strong>the</strong>ring and<br />
understanding information about <strong>the</strong> world, all <strong>of</strong> this<br />
clearly constitutes significant empirical research.<br />
Bowman (1963) presents an excellent example <strong>of</strong><br />
learning through implementation. The paper was<br />
already a classic by <strong>the</strong> late 1960s when I was a graduate<br />
student, and it pr<strong>of</strong>oundly influenced me and<br />
many o<strong>the</strong>rstudents at that time. Holt et al. (1955)<br />
describe decision rules to determine optimal monthly<br />
production, inventory, and workforce levels as linearfunctions<br />
<strong>of</strong> priorvalues <strong>of</strong> <strong>the</strong>se variables and a<br />
demand forecast. Bowman (1983) applied <strong>the</strong>se rules<br />
to ice cream, chocolate, and candy plants in three separate<br />
studies. He first used accounting data to derive<br />
accurate estimates <strong>of</strong> <strong>the</strong> coefficients required in <strong>the</strong><br />
decision rules and obtained results that improved on<br />
existing practice. But, surprisingly, he got still better<br />
results by choosing <strong>the</strong>se coefficients so <strong>the</strong> output <strong>of</strong><br />
<strong>the</strong> decision rules most closely matched, on average,<br />
managers’ past decisions. Amazingly, rules fitted to<br />
managers’ prior decisions did better than <strong>the</strong> managers<br />
<strong>the</strong>mselves!<br />
Bowman’s findings illustrate that one <strong>of</strong>ten makes<br />
interesting discoveries in <strong>the</strong> process <strong>of</strong> implementing<br />
an algorithm. Bowman’s discovery was a principle<br />
that he summarized as follows. Managers make<br />
good decisions on average, but <strong>the</strong>y are hurt by variation<br />
in <strong>the</strong>irdecision making. Thus, “A decision rule<br />
with mean coefficients estimated from management’s<br />
behaviorshould be betterthan actual performance [<strong>of</strong><br />
<strong>the</strong> managers]. It may also be better than a rule with<br />
coefficients supplied by traditional analysis” (p. 321).<br />
3. <strong>Operations</strong> <strong>Management</strong><br />
Econometrics<br />
Many fields apply regression and o<strong>the</strong>r statistical<br />
analysis tools to data sets in an attempt to discover<br />
evidence to support various hypo<strong>the</strong>ses. In fact, this<br />
is <strong>the</strong> type <strong>of</strong> research that most people in our field<br />
associate with <strong>the</strong> term empirical research.<br />
Without question, <strong>the</strong> International Motor Vehicle<br />
Program study <strong>of</strong> 70 automobile assembly plants<br />
worldwide is <strong>the</strong> “mo<strong>the</strong>r” <strong>of</strong> all operations management<br />
econometrics research efforts. This ambitious<br />
study, which began at MIT. in <strong>the</strong> late 1980s and<br />
continues to this day, seeks to discoverand validate<br />
management practices associated with high levels <strong>of</strong><br />
quality and productivity in an automobile assembly<br />
plant. The average number<strong>of</strong> defects pervehicle as<br />
measured by <strong>the</strong> J. D. Powers survey that tests most<br />
car models is used as <strong>the</strong> measure <strong>of</strong> quality. Productivity<br />
is equated to <strong>the</strong> number<strong>of</strong> hours to assemble<br />
a vehicle, a metric tracked by most automobile<br />
plants. To compare two different plants, this metric<br />
is adjusted to normalize for differences in vehicle<br />
complexity.<br />
The study 2 found large differences in quality and<br />
productivity between plants. Moreover, those plants<br />
with <strong>the</strong> highest productivity also tended to have <strong>the</strong><br />
highest quality, contradicting <strong>the</strong> conventional wisdom<br />
that productivity and quality trade <strong>of</strong>f against<br />
each o<strong>the</strong>r. High performance was correlated with<br />
certain practices that have come to be called “lean<br />
production” and include just-in-time inventory management<br />
and a high degree <strong>of</strong> worker involvement.<br />
Although <strong>the</strong>se practices are typically associated with<br />
Japanese manufacturers, it was shown that this is<br />
not merely a Japanese effect; <strong>the</strong>re were a significant<br />
number <strong>of</strong> poorly performing plants within Japan<br />
that did not follow lean production principles and<br />
an equally significant number<strong>of</strong> highly performing<br />
plants outside <strong>of</strong> Japan that did follow lean production<br />
principles.<br />
The automobile industry has proven to be a fertile<br />
context for<strong>the</strong> application <strong>of</strong> econometric methods.<br />
Ano<strong>the</strong>rhighly successful example is Clark and<br />
Fujimoto (1991), which reports results <strong>of</strong> an extensive<br />
study <strong>of</strong> product development in <strong>the</strong> auto industry.<br />
The authors examined a large number <strong>of</strong> new model<br />
development projects at 20 automobile manufacturers<br />
worldwide to understand <strong>the</strong> management practices<br />
that influenced design quality, product development<br />
time, and product development productivity as measured<br />
by <strong>the</strong> engineering hours required by a project.<br />
2 The results <strong>of</strong> this ongoing study have been published in many<br />
papers. Early work can be found in Krafcik (1988) and MacDuffie<br />
(1991), with more recent results in MacDuffie et al. (1996).
Fisher: <strong>Streng<strong>the</strong>ning</strong> <strong>the</strong> <strong>Empirical</strong> <strong>Base</strong> <strong>of</strong> <strong>Operations</strong> <strong>Management</strong><br />
Manufacturing & Service <strong>Operations</strong> <strong>Management</strong> 9(4), pp. 368–382, © 2007 INFORMS 373<br />
Table 1 Impact <strong>of</strong> Variety on Supply Chain Costs<br />
Impact on Impact on<br />
production market Ideal supply<br />
Type <strong>of</strong> variety costs mediation costs chain<br />
Material High Low Large, centralized<br />
plant<br />
Frame geometry Low High Local<br />
and size<br />
Colors Low High Local<br />
Components Low High Local<br />
They found big differences between companies and<br />
regions, although generally <strong>the</strong> Japanese companies<br />
were more effective on all dimensions, producing<br />
higher-quality designs with fewer engineering hours<br />
in about half <strong>the</strong> lead time <strong>of</strong> <strong>the</strong> U.S. companies.<br />
They coined <strong>the</strong> term “heavy-weight project manager”<br />
for <strong>the</strong> management style used in <strong>the</strong>se projects,<br />
a concept that has become pervasive in product development<br />
management.<br />
Writing about bicycles, not cars, Randall and Ulrich<br />
(2001) sought to understand <strong>the</strong> relationship between<br />
product variety and supply chain structure in <strong>the</strong><br />
world bicycle industry. As shown in Table 1, <strong>the</strong><br />
authors identified four types <strong>of</strong> bicycle variety and<br />
conjectured as to <strong>the</strong>ir impact on production costs and<br />
market mediation costs, <strong>the</strong> cost <strong>of</strong> mismatch between<br />
supply and demand. <strong>Base</strong>d on this, <strong>the</strong>y hypo<strong>the</strong>sized<br />
as to <strong>the</strong> ideal supply chain fora company with a<br />
given level <strong>of</strong> each type <strong>of</strong> variety. They <strong>the</strong>n analyzed<br />
public and survey data from 48 firms comprising<br />
70% <strong>of</strong> <strong>the</strong> bicycle industry to determine <strong>the</strong>ir level <strong>of</strong><br />
variety, supply chain structure, and return on assets.<br />
They found that 71% <strong>of</strong> <strong>the</strong> firms used <strong>the</strong>ir hypo<strong>the</strong>sized<br />
appropriate supply chain, and those firms that<br />
did had a higherreturn on assets than those that<br />
did not.<br />
4. Case Studies<br />
We should not underestimate <strong>the</strong> value <strong>of</strong> less structured<br />
empiricism. Something as simple as a conversation<br />
with a manageroverlunch can be extremely<br />
useful in identifying problems and hypo<strong>the</strong>ses for fur<strong>the</strong>rinvestigation,<br />
especially if a series <strong>of</strong> <strong>the</strong>se conversations<br />
over time all point in <strong>the</strong> same direction.<br />
The preparation for writing a case usually includes<br />
numerous relatively unstructured conversations with<br />
managers. Cases are written both to support teaching<br />
and directly for research. Because <strong>the</strong>y document<br />
a particular operations issue in a single company,<br />
<strong>the</strong>y are also a wonderful way to begin to formulate<br />
research problems and hypo<strong>the</strong>ses.<br />
Jaikumar(1986) provides a good example <strong>of</strong> how a<br />
case can reveal research issues. A description <strong>of</strong> this<br />
case and its research implications was provided previously<br />
in Fisher(1991), and <strong>the</strong> description here is<br />
based on that paper.<br />
As described in <strong>the</strong> case, <strong>the</strong> maintenance <strong>of</strong> turbine<br />
generators is a major challenge for <strong>the</strong> U.S. electric<br />
power industry because 40% <strong>of</strong> its generators are<br />
more than 20 years old, and even a single day <strong>of</strong><br />
down time can cost as much as $500,000. In this environment,<br />
it is vital for a utility to be able to detect<br />
that a generator is about to crash, because this allows<br />
that utility to bring <strong>the</strong> generator down “s<strong>of</strong>tly” and<br />
fix <strong>the</strong> problem more quickly and less expensively<br />
than if <strong>the</strong> generator crashed without warning. For<br />
this reason, utilities have armed <strong>the</strong>ir generators with<br />
hundreds <strong>of</strong> on-line sensors to measure in real time<br />
voltage, temperature, pressure, and o<strong>the</strong>r key variables.<br />
But <strong>the</strong>n <strong>the</strong> utilities face <strong>the</strong> problem <strong>of</strong> making<br />
sense out <strong>of</strong> all this data.<br />
It was to satisfy this need that <strong>the</strong> steam turbine<br />
division <strong>of</strong> Westinghouse Electric developed an expert<br />
system that can continuously monitordata from up to<br />
110 turbine sensors, looking for patterns that signal<br />
trouble. The system uses a base <strong>of</strong> 1,300 rules<br />
that were developed in consultation with a number<strong>of</strong><br />
Westinghouse engineers. They identified 350<br />
failure conditions corresponding to different components<br />
within a generator that can fail. At any time,<br />
if <strong>the</strong> system detects that something is amiss, it can<br />
recommend that personnel at <strong>the</strong> generator site perform<br />
additional confirming tests while <strong>the</strong> generator<br />
is running. Depending on <strong>the</strong> results <strong>of</strong> <strong>the</strong>se tests,<br />
<strong>the</strong> system can <strong>the</strong>n recommend that a generator be<br />
stopped, inspected, and fixed if necessary. The system<br />
has been used since 1984 by <strong>the</strong> Texas Utilities<br />
Generating Company and is credited with a number<br />
<strong>of</strong> “early warnings” on imminent failures that have<br />
saved millions <strong>of</strong> dollars in down time.<br />
One interesting research question motivated by this<br />
case is whe<strong>the</strong>ra modeling/algorithmic approach<br />
could be developed forproblems <strong>of</strong> this type that
Fisher: <strong>Streng<strong>the</strong>ning</strong> <strong>the</strong> <strong>Empirical</strong> <strong>Base</strong> <strong>of</strong> <strong>Operations</strong> <strong>Management</strong><br />
374 Manufacturing & Service <strong>Operations</strong> <strong>Management</strong> 9(4), pp. 368–382, © 2007 INFORMS<br />
would be more effective than an expert system.<br />
Clearly, <strong>the</strong> work at Westinghouse has structured <strong>the</strong><br />
problem to <strong>the</strong> point where one can visualize applying<br />
ideas like those in Ross (1971). Forexample, it is<br />
tempting to think <strong>of</strong> <strong>the</strong> 350 identified failure conditions<br />
as states in a Markov process and <strong>the</strong> actions<br />
available to <strong>the</strong> expert as <strong>the</strong> actions in Ross’s model.<br />
However, <strong>the</strong> scale <strong>of</strong> real problems is several orders<br />
<strong>of</strong> magnitude bigger than <strong>the</strong> problems addressed in<br />
<strong>the</strong> literature, so computational tractability could be a<br />
majorchallenge.<br />
5. Principles<br />
Prescriptions derived from empirical research can be<br />
numeric and detailed, such as <strong>the</strong> output <strong>of</strong> an algorithm,<br />
or more qualitative principles that provide general<br />
guidance but require some interpretation on <strong>the</strong><br />
part <strong>of</strong> <strong>the</strong> user. Perhaps <strong>the</strong> best example <strong>of</strong> prescriptive<br />
principles is <strong>the</strong> Toyota Production System, which<br />
provides guidance on how to structure a production<br />
process and manage workers to achieve a high level<br />
<strong>of</strong> quality and productivity.<br />
My favorite example <strong>of</strong> principles reported in <strong>the</strong><br />
academic literature is <strong>the</strong> study by Jordan and Graves<br />
(1995), which describes several principles <strong>of</strong> manufacturing<br />
flexibility derived from empirical research<br />
within <strong>the</strong> auto industry. They consider a company<br />
that makes N products with uncertain demand in M<br />
factories with fixed capacity. Because <strong>of</strong> <strong>the</strong> limited<br />
capacity, it may not be possible to satisfy fully a particulardemand<br />
realization. The amount <strong>of</strong> demand<br />
that is satisfied can be increased by providing <strong>the</strong><br />
flexibility to make a particular product in more than<br />
one plant. In <strong>the</strong> extreme, when every product can<br />
be made in all plants, demand can be fully satisfied<br />
as long as total demand for<strong>the</strong> N products does<br />
not exceed <strong>the</strong> total capacity <strong>of</strong> <strong>the</strong> M plants. However,<br />
providing this maximal level <strong>of</strong> flexibility is usually<br />
prohibitively expensive, so Jordan and Graves<br />
(1995) ask how closely more limited flexibility would<br />
come to achieving <strong>the</strong> demand satisfaction benefits<br />
<strong>of</strong> full flexibility. Starting with this natural question<br />
and drawing on insights gleaned from extensive interaction<br />
with General Motors managers, <strong>the</strong>y derive<br />
several principles <strong>of</strong> flexibility and provide analytic<br />
support for <strong>the</strong>se principles.<br />
Their sharpest result is stated as a property <strong>of</strong> <strong>the</strong><br />
product-plant assignment graph, which has a node<br />
foreach product and foreach plant and an arc connecting<br />
product node i with plant node j if product i<br />
can be made in plant j. They show that when M = N ,<br />
<strong>the</strong> ability to make each product in just two plants<br />
provides nearly <strong>the</strong> same level <strong>of</strong> demand satisfaction<br />
as full flexibility, provided <strong>the</strong> product-plant assignment<br />
graph is connected.<br />
6. Integrating Theory and Empirics<br />
As discussed at <strong>the</strong> start <strong>of</strong> this paper, medicine,<br />
physics, and finance have benefited from a healthy<br />
synergy between <strong>the</strong>oretical and empirical research.<br />
Theoreticians develop hypo<strong>the</strong>ses that are verified<br />
by empiricists, and empiricists identify interesting<br />
phenomena in <strong>the</strong> world to be explained through<br />
additional <strong>the</strong>ories. Some examples <strong>of</strong> research that<br />
integrates <strong>the</strong>ory and empiricism are beginning to<br />
emerge in our field.<br />
One such example focuses on a betterunderstanding<br />
<strong>of</strong> <strong>the</strong> relationship among <strong>the</strong> four traditional<br />
performance measures in operations management:<br />
cost, manufacturing conformance quality, delivery<br />
speed, and flexibility. Traditionally, <strong>the</strong>se measures<br />
have been thought to trade <strong>of</strong>f against each o<strong>the</strong>r.<br />
For example, improving quality meant increasing cost<br />
(we will use cost and quality as ourexamples in<br />
<strong>the</strong> rest <strong>of</strong> this section, although it should be clear<br />
that <strong>the</strong> concept is also applicable to <strong>the</strong> o<strong>the</strong>rperformance<br />
measures). More recently, Clark (1996) and<br />
Hayes and Pisano (1996) have suggested that companies<br />
can and do follow improvement paths that simultaneously<br />
increase quality and reduce cost.<br />
A little thought makes it clearthat a company<br />
can take various actions to improve quality; some <strong>of</strong><br />
those actions will increase cost and some will reduce<br />
cost. To illustrate, consider a process that produces<br />
a product with a 10% defect rate. An inspector at<br />
<strong>the</strong> end <strong>of</strong> <strong>the</strong> process attempts to cull out defective<br />
units to be reworked or scrapped. But because<br />
inspection is imperfect, <strong>the</strong> inspector only catches half<br />
<strong>the</strong> defects, resulting in a 5% defect rate reaching<br />
<strong>the</strong> market. Adding a second inspector who independently<br />
detects half <strong>the</strong> remaining defects would<br />
reduce <strong>the</strong> defect rate to 2.5% but would increase cost<br />
because <strong>of</strong> <strong>the</strong> expense <strong>of</strong> <strong>the</strong> additional inspector,
Fisher: <strong>Streng<strong>the</strong>ning</strong> <strong>the</strong> <strong>Empirical</strong> <strong>Base</strong> <strong>of</strong> <strong>Operations</strong> <strong>Management</strong><br />
Manufacturing & Service <strong>Operations</strong> <strong>Management</strong> 9(4), pp. 368–382, © 2007 INFORMS 375<br />
rework, and scrap. We could continue to add inspectors,<br />
steadily reducing <strong>the</strong> defect rate while increasing<br />
cost and thus moving along a cost-quality trade<strong>of</strong>f<br />
curve. On <strong>the</strong> o<strong>the</strong>rhand, if we identify one ormore<br />
root causes <strong>of</strong> defects and modify <strong>the</strong> process to<br />
remove <strong>the</strong>se root causes, <strong>the</strong>n <strong>the</strong> process will produce<br />
fewerdefective units so we will both lower<strong>the</strong><br />
defect rate and reduce cost because <strong>of</strong> reduced rework<br />
and scrap. Total cost will be reduced as long as <strong>the</strong><br />
cost <strong>of</strong> diagnosing and fixing root causes <strong>of</strong> defects is<br />
less than <strong>the</strong> savings in rework and scrap expenses.<br />
Total Quality <strong>Management</strong> (TQM) wisdom suggests<br />
sorting root causes on <strong>the</strong> number <strong>of</strong> product defects<br />
<strong>the</strong>y cause and fixing first those root causes responsible<br />
for <strong>the</strong> most product defects. If this approach is<br />
followed, we will likely improve quality and reduce<br />
cost as we attack <strong>the</strong> most serious process defects initially;<br />
eventually we will be fixing process defects that<br />
cause so few product defects that <strong>the</strong> savings in rework<br />
and scrap will be less than <strong>the</strong> cost <strong>of</strong> process<br />
improvement—<strong>the</strong>n improved quality comes at <strong>the</strong><br />
expense <strong>of</strong> increased cost.<br />
Several authors have suggested principles that<br />
align with <strong>the</strong>se observations. Clark (1996), Hayes and<br />
Pisano (1996), and Porter (1996) observe that most<br />
companies operate <strong>of</strong>f <strong>the</strong> efficient frontier between<br />
quality and cost and are <strong>the</strong>refore initially able to<br />
improve on both dimensions. But eventually <strong>the</strong>y<br />
reach <strong>the</strong> frontier and face a trade<strong>of</strong>f between cost and<br />
quality. Ferdows and DeMeyer (1990) suggest that<br />
companies should and do give priority to quality over<br />
cost. If <strong>the</strong>y are <strong>of</strong>f <strong>the</strong> efficient frontier, <strong>the</strong>n <strong>the</strong>y<br />
improve both quality and cost but give higher priority<br />
to quality improvement. If <strong>the</strong>y face a trade<strong>of</strong>f<br />
between quality and cost, <strong>the</strong>n <strong>the</strong> improved quality<br />
is at <strong>the</strong> expense <strong>of</strong> increased cost. Once <strong>the</strong>y reach<br />
a position <strong>of</strong> high quality, <strong>the</strong>y may <strong>the</strong>n focus on<br />
cost reduction and eventually move to a new efficient<br />
frontier in which both quality and cost are improved.<br />
Lapre and Scudder (2004) examine <strong>the</strong>se hypo<strong>the</strong>ses<br />
within <strong>the</strong> context <strong>of</strong> <strong>the</strong> airline industry using<br />
public data collected and reported by <strong>the</strong> U.S. Department<br />
<strong>of</strong> Transportation (DOT). Their quality metric is<br />
<strong>the</strong> number<strong>of</strong> customercomplaints made to <strong>the</strong> DOT<br />
per100,000 passengers, <strong>the</strong>ircost measure is cost per<br />
seat-mile flown, and <strong>the</strong>irasset utilization measure is<br />
fleet utilization, <strong>the</strong> average percentage <strong>of</strong> <strong>the</strong> time in<br />
a 24-hourday that a plane was available forservice<br />
that <strong>the</strong> plane was in active use; a period <strong>of</strong> active<br />
use is defined from when <strong>the</strong> plane first moves under<br />
its own power from <strong>the</strong> boarding ramp at <strong>the</strong> departure<br />
airport until it comes to rest at <strong>the</strong> ramp for <strong>the</strong><br />
destination airport. Using DOT data, <strong>the</strong>se three variables<br />
were tabulated for each <strong>of</strong> <strong>the</strong> 11 years from<br />
1988–1998 for<strong>the</strong> 10 majorairlines operating during<br />
this time: Alaska, America West, American, Continental,<br />
Delta, Northwest, Southwest, TWA, United,<br />
and U.S. Airways. The study qualitatively examined<br />
<strong>the</strong> cost-quality path followed by each airline over<br />
<strong>the</strong> 11 years and concluded that when an airline was<br />
forced to make a trade<strong>of</strong>f between quality and cost,<br />
it generally elected first to improve quality at <strong>the</strong><br />
expense <strong>of</strong> cost, and, in some instances, subsequently<br />
also improved cost to arrive at an overall superior<br />
position. Their empirical research thus provides support<br />
for <strong>the</strong> hypo<strong>the</strong>ses <strong>of</strong> Clark (1996), Ferdows and<br />
DeMeyer(1990), Hayes and Pisano (1996), and Porter<br />
(1996).<br />
As ano<strong>the</strong>r example: <strong>of</strong> integration <strong>of</strong> <strong>the</strong>ory and<br />
empirics, Cachon and Lariviere (2001) and Terwiesch<br />
et al. (2005) both considera demand planning problem<br />
between a manufacturer and supplier, <strong>the</strong> first<br />
from a <strong>the</strong>oretical and <strong>the</strong> second from an empirical<br />
perspective. A manufacturer is launching a new product<br />
and will be purchasing a specialized key component<br />
from a supplier. In advance <strong>of</strong> <strong>the</strong> product<br />
launch, <strong>the</strong> supplierneeds to build capacity, which<br />
can only be used for<strong>the</strong> specialized key component.<br />
To help <strong>the</strong> supplierdecide what level <strong>of</strong> capacity to<br />
build, <strong>the</strong> manufacturer provides <strong>the</strong> supplier with its<br />
forecast <strong>of</strong> demand in <strong>the</strong> form <strong>of</strong> a probability density<br />
function <strong>of</strong> demand. Note that <strong>the</strong> manufacturer<br />
has an incentive to inflate its forecast to encourage<br />
<strong>the</strong> supplierto build ample capacity and thus minimize<br />
<strong>the</strong> risk that it might loose business because <strong>of</strong><br />
inadequate supply. The supplier, on <strong>the</strong> o<strong>the</strong>r hand,<br />
knows <strong>the</strong> manufacturer has this bias and is <strong>the</strong>refore<br />
inclined to discount <strong>the</strong> forecast.<br />
This is a very real problem. As one personal example,<br />
I worked once with <strong>the</strong> manufacturing division<br />
<strong>of</strong> a large company that had tracked <strong>the</strong> forecasts prepared<br />
by <strong>the</strong> sales and marketing division and found<br />
<strong>the</strong>y exceeded actual demand by 30%. Consequently,<br />
<strong>the</strong>y began to divide <strong>the</strong> sales and marketing forecast
Fisher: <strong>Streng<strong>the</strong>ning</strong> <strong>the</strong> <strong>Empirical</strong> <strong>Base</strong> <strong>of</strong> <strong>Operations</strong> <strong>Management</strong><br />
376 Manufacturing & Service <strong>Operations</strong> <strong>Management</strong> 9(4), pp. 368–382, © 2007 INFORMS<br />
by 1.3 for planning purposes. Sales and marking got<br />
wind <strong>of</strong> this, and you can guess what <strong>the</strong>y did; <strong>the</strong>y<br />
began to inflate <strong>the</strong>irforecasts by 60%!<br />
Cachon and Lariviere (2001) consider this complex<br />
planning context and ask how a manufacturer that is<br />
telling <strong>the</strong> truth about its forecast can convince <strong>the</strong><br />
supplier<strong>of</strong> this. They design contracts <strong>the</strong> manufacturercan<br />
<strong>of</strong>fer<strong>the</strong> supplierthat would not be attractive<br />
to <strong>the</strong> manufacturer if <strong>the</strong> true forecast were less<br />
than <strong>the</strong>y were representing.<br />
Building in part on this research, Terwiesch et al.<br />
(2005) take an empirical approach to this problem.<br />
They worked with a manufacturer and 78 <strong>of</strong> <strong>the</strong>ir<br />
suppliers to collect data on 3,000 instances over two<br />
years when <strong>the</strong> manufacturer shared a forecast with<br />
a supplier. They note that <strong>the</strong> problem is <strong>of</strong>ten a<br />
repeated game, so when a supplier receives a forecast<br />
from a manufacturer, its faith in that forecast<br />
will depend on <strong>the</strong> accuracy <strong>of</strong> all <strong>of</strong> <strong>the</strong> forecasts<br />
received from that manufacturer in <strong>the</strong> past. Those<br />
suppliers that have received forecasts that were relatively<br />
poor(biased high and/orchanged frequently in<br />
<strong>the</strong> past) provided significantly worse service, delivering<br />
less than was ordered and delivering it late. Conversely,<br />
<strong>the</strong> manufacturer tended to inflate its forecast<br />
to <strong>the</strong> extent that it had been short shipped by<br />
<strong>the</strong> supplierin <strong>the</strong> past, thus creating <strong>the</strong> conditions<br />
<strong>of</strong> what would appearto be a downwardly spiraling<br />
relationship.<br />
Experimentation is a common form <strong>of</strong> empirical<br />
research in <strong>the</strong> physical sciences and, as described in<br />
Croson and Donohue (2002), is emerging as a useful<br />
technique in operations management. Probably<br />
<strong>the</strong> most famous laboratory experiment in operations<br />
management is <strong>the</strong> beergame described in Sterman<br />
(1989). Participants play <strong>the</strong> role <strong>of</strong> managers <strong>of</strong> firms<br />
in a beersupply chain, comprised <strong>of</strong> a manufacturer,<br />
a wholesaler, a distributor, and a retailer. They<br />
make supply decisions based on recent downstream<br />
demand or orders, but with no knowledge <strong>of</strong> future<br />
demand ororders. The beergame has been used<br />
almost universally in courses on supply chain management<br />
and has been <strong>the</strong> source <strong>of</strong> an important conjecture<br />
about supply chains called <strong>the</strong> bullwhip effect.<br />
It is usually observed in <strong>the</strong> beergame that ordervariability<br />
increases as one moves upstream in <strong>the</strong> supply<br />
chain, just as <strong>the</strong> movement <strong>of</strong> a bullwhip increases<br />
from <strong>the</strong> handle to <strong>the</strong> tip. For example, <strong>the</strong> variation<br />
in manufacturer orders is usually much greater than<br />
retail demand.<br />
Formany years it was believed, and anecdotally<br />
observed, that real supply chains exhibited this same<br />
phenomenon. Then Lee et al. (1997) developed analytic<br />
results for various supply chain planning contexts<br />
that would explain why <strong>the</strong> bullwhip effect<br />
could be expected to occur. Guided by this framework,<br />
Cachon et al. (2007) used data from <strong>the</strong> U.S.<br />
Census Bureau and <strong>the</strong> Bureau <strong>of</strong> Economic Analysis<br />
on sales, inventory, and prices to search for instances<br />
<strong>of</strong> <strong>the</strong> bullwhip effect. They found <strong>the</strong> bullwhip effect<br />
in some situations but not in o<strong>the</strong>rs, which led <strong>the</strong>m<br />
to develop a more refined framework <strong>of</strong> <strong>the</strong> factors<br />
that increase demand variability and those that attenuate<br />
demand variability as one moves upstream in a<br />
supply chain.<br />
The research reviewed in this section that was conducted<br />
using both <strong>the</strong>orizing and empirical research,<br />
with each stimulating <strong>the</strong> o<strong>the</strong>r, has many positive<br />
features. However, some might argue that our field<br />
lacks a cohesive and general <strong>the</strong>ory <strong>of</strong> operations,<br />
and <strong>the</strong>refore any discussion <strong>of</strong> integration <strong>of</strong> <strong>the</strong>ory<br />
and empirics must be postponed until we have<br />
such a <strong>the</strong>ory. I would argue that, as suggested by<br />
von Neuman (1956), <strong>the</strong> best <strong>the</strong>ories are <strong>the</strong> result<br />
<strong>of</strong> efforts to understand real phenomenon: thus, <strong>the</strong>orizing<br />
based on empirics increases <strong>the</strong> chances <strong>of</strong><br />
improving <strong>the</strong> <strong>the</strong>oretical base <strong>of</strong> operations management.<br />
O<strong>the</strong>rs have suggested that relations between<br />
<strong>the</strong>oreticians and empiricists are <strong>of</strong>ten contentious in<br />
o<strong>the</strong>rfields, including medicine, physics, and finance.<br />
I would counterthat vigorous debate about issues is<br />
<strong>the</strong> mark <strong>of</strong> a healthy field. Doing research on important<br />
issues that people care about will always invite<br />
controversy, but that’s a good thing, not a bad thing.<br />
7. Data Sources<br />
Data are <strong>the</strong> raw materials <strong>of</strong> empirical research, so a<br />
crucial question for an empirical researcher is where<br />
to get data. I examined a number<strong>of</strong> papers on empirical<br />
research in operations management to identify<br />
data sources. The results are compiled in Table 2.<br />
Many <strong>of</strong> <strong>the</strong> papers cited as examples <strong>of</strong> various types
Fisher: <strong>Streng<strong>the</strong>ning</strong> <strong>the</strong> <strong>Empirical</strong> <strong>Base</strong> <strong>of</strong> <strong>Operations</strong> <strong>Management</strong><br />
Manufacturing & Service <strong>Operations</strong> <strong>Management</strong> 9(4), pp. 368–382, © 2007 INFORMS 377<br />
Table 2 Data Sources<br />
Data source Examples<br />
Public accounting and o<strong>the</strong>r public data: Annual reports, Compustat, Trade Cachon and Olivares (2007), Cachon et al. (2007), Fisher et al. (1999),<br />
and Industry Index, Dow Jones News Service, Department <strong>of</strong> Transportation, Gaur et al. (2005), Hendricks and Singhal (1997), Hendricks and<br />
U.S. Census Bureau, Bureau <strong>of</strong> Economic Analysis, and o<strong>the</strong>r similar<br />
sources<br />
Singhal (2003), Lapre and Scudder (2004)<br />
Private accounting data Datar et al. (1997), DeHoratius and Raman (2007), Mukherjee et al. (1998)<br />
Data created by <strong>the</strong> researchers from company records or survey <strong>of</strong> Clark and Fujimoto (1991), DeHoratius and Raman (2007), Jaikumar<br />
managers (1986), Jordan and Graves (1995), Khanna and Iansiti (1997), Krafcik<br />
(1988), Macduffie (1991), MacDuffie et al. (1996), Randall and Ulrich<br />
(2001), Sterman et al. (1997), Terwiesch et al. (2005), Ton and Raman<br />
(2005)<br />
Direct observation <strong>of</strong> products Ulrich and Pearson (1998)<br />
Direct observation <strong>of</strong> processes augmented by discussions with managers MacDuffie (1997), Tucker (2004)<br />
Experiments in a “lab” Schweitzer and Cachon (2000), Sterman (1989)<br />
Experiments in companies Bartholdi et al. (1983), Bowman (1963), Burchill and<br />
Fine (1997), Lapre and Van Wassenhove (2001)<br />
<strong>of</strong> data have been discussed already in this paper.<br />
O<strong>the</strong>rs are new and are discussed in this section.<br />
It is readily apparent from Table 2 that <strong>the</strong>re are<br />
many sources for data. Happily, much useful data are<br />
available in <strong>the</strong> public domain. Annual reports and<br />
o<strong>the</strong>rpublic accounting data published by companies<br />
are <strong>the</strong> most obvious example, but <strong>the</strong>re are many<br />
o<strong>the</strong>r sources <strong>of</strong> public data. As already discussed,<br />
Lapre and Scudder (2004) found rich data on airlines<br />
available through <strong>the</strong> DOT, which <strong>the</strong>y were able to<br />
use to analyze improvements in quality and productivity.<br />
Cachon et al. (2007) used data from <strong>the</strong> U.S.<br />
Census Bureau and <strong>the</strong> Bureau <strong>of</strong> Economic Analysis<br />
on sales, inventory, and prices in <strong>the</strong>ir search<br />
for<strong>the</strong> bullwhip effect. Gauret al. (2005) used Standard<br />
and Poor’s Compustat database to construct a<br />
data set <strong>of</strong> inventory, total assets, current assets, and<br />
o<strong>the</strong>rvariables for311 public retailers for<strong>the</strong> period<br />
1985–2000 and used it to explain variation in inventory<br />
turnover using gross margin, capital intensity,<br />
and sales surprise. Hendricks and Singhal (1997) used<br />
news articles from <strong>the</strong> Trade and Industry Index and<br />
<strong>the</strong> Dow Jones News Service to identify instances in<br />
which companies were late to market with new products<br />
and <strong>the</strong>n measured <strong>the</strong> impact this had on share<br />
price. Hendricks and Singhal (2003) followed a similar<br />
approach, using articles from <strong>the</strong> Dow Jones News<br />
Service and <strong>the</strong> Wall Street Journal to identify instances<br />
<strong>of</strong> supply disruption to measure <strong>the</strong> impact that had<br />
on share prices.<br />
It is worth noting that some public data exist not as<br />
a database but as transaction data created for ano<strong>the</strong>r<br />
purpose that can be assembled by a researcher into a<br />
database to support specific research. In <strong>the</strong>ir study<br />
<strong>of</strong> parts sharing in <strong>the</strong> auto industry, Fisher et al.<br />
(1999) needed to know which brake components were<br />
shared across which cars. To assemble this information,<br />
<strong>the</strong>y used a data service created by a third-party<br />
provider that is consulted by auto salvage yards to<br />
determine if a brake salvaged from car A can be used<br />
in carB. Ano<strong>the</strong>rintriguing example is provided by<br />
<strong>the</strong> Cachon and Olivares (2007) study <strong>of</strong> competition<br />
among auto dealers. To facilitate customer shopping,<br />
General Motors auto dealers provide real-time information<br />
over<strong>the</strong> Internet <strong>of</strong> <strong>the</strong>irinventory availability<br />
by Vehicle Identification Number(VIN). Cachon and<br />
Olivares (2007) programmed and ran a Web crawler<br />
against this data service nightly and were able daily<br />
to construct information on dealer sales, transfers,<br />
receipts, and inventory.<br />
In addition to <strong>the</strong> accounting results <strong>the</strong>y report<br />
publicly, all companies maintain substantial databases<br />
for internal managerial purposes and can <strong>of</strong>ten be<br />
persuaded to provide this data for research purposes<br />
in exchange forassurances formaintaining confidentiality<br />
<strong>of</strong> sensitive information. The virtue <strong>of</strong> <strong>the</strong>se<br />
“private accounting data” is that <strong>the</strong>y already exist<br />
and hence can be shared by companies with relatively<br />
little effort. Datar et al. (1997) worked with three<br />
companies and “obtained critical information on <strong>the</strong>
Fisher: <strong>Streng<strong>the</strong>ning</strong> <strong>the</strong> <strong>Empirical</strong> <strong>Base</strong> <strong>of</strong> <strong>Operations</strong> <strong>Management</strong><br />
378 Manufacturing & Service <strong>Operations</strong> <strong>Management</strong> 9(4), pp. 368–382, © 2007 INFORMS<br />
progress <strong>of</strong> new product development from company<br />
records ��� � We retained 220 new products for which<br />
complete histories on time to prototype, time to volume<br />
production, and engineering expenditures were<br />
available.” They used <strong>the</strong>se data to identify features<br />
<strong>of</strong> new product development structures that correlate<br />
with a short time to market.<br />
DeHoratius and Raman (2004) worked with a retailer<br />
that audited its stores to identify discrepancies<br />
between <strong>the</strong> quantity <strong>of</strong> a SKU as counted on <strong>the</strong><br />
shelf in <strong>the</strong> store compared with <strong>the</strong> same quantity as<br />
listed in corporate records. They analyzed <strong>the</strong> data to<br />
understand <strong>the</strong> level and drivers <strong>of</strong> inventory record<br />
accuracy.<br />
Mukherjee et al. (1998) used company data on 62<br />
quality improvement projects conducted by N. V.<br />
Bekaert, S.A., <strong>the</strong> world’s largest supplier <strong>of</strong> steel<br />
wire, to determine <strong>the</strong> impact <strong>the</strong>se projects had on<br />
<strong>the</strong> way <strong>the</strong> organization learned.<br />
Existing company data have <strong>the</strong> advantage <strong>of</strong> being<br />
easy to assemble, but those data might not have<br />
all <strong>the</strong> information needed to answer <strong>the</strong> research<br />
questions being addressed, so researchers will <strong>of</strong>ten<br />
augment existing data with additional data collected<br />
for <strong>the</strong> purpose <strong>of</strong> <strong>the</strong>ir research project. As mentioned<br />
previously, Clark and Fujimoto (1991), Jaikumar<br />
(1986), Jordan and Graves (1995), Krafcik (1988),<br />
Macduffie (1991), MacDuffie et al. (1996), and Terwiesch<br />
et al. (2005) all provide examples <strong>of</strong> researchers<br />
constructing a database within a company.<br />
Ano<strong>the</strong>rexample is provided by DeHoratius and<br />
Raman (2007), who worked with an audio electronics<br />
retailer that had changed <strong>the</strong> store manager incentives<br />
<strong>of</strong> ano<strong>the</strong>r retailer it had acquired to reduce <strong>the</strong> incentive<br />
<strong>the</strong> store managers had to minimize inventory<br />
shrink. They found that shrink did indeed increase,<br />
but <strong>the</strong> cost <strong>of</strong> this increase was more than <strong>of</strong>fset by<br />
<strong>the</strong> pr<strong>of</strong>it on increased sales, which resulted because<br />
activities that reduce shrink tend to also reduce sales.<br />
Khanna and Iansiti (1997, p. 413) worked with all <strong>the</strong><br />
mainframe computer manufacturers in <strong>the</strong> world and<br />
“collected observations on all major multichip module<br />
related projects ���through multiple interviews with<br />
<strong>the</strong> key managers and engineers as well as through<br />
questionnaires” to better understand how <strong>the</strong>se firms<br />
allocated resources during different stages <strong>of</strong> a development<br />
project. Sterman et al. (1997) sought to understand<br />
why financial performance at Analog Devices<br />
worsened after a dramatically successful Total Quality<br />
<strong>Management</strong> program that doubled yield, cut cycle<br />
time in half, and reduced defects by an order <strong>of</strong> magnitude.<br />
To do this, <strong>the</strong>y “used econometric estimation,<br />
interviews, observation, and archival data to specify<br />
and estimate” (p. 503) <strong>the</strong> parameters <strong>of</strong> a simulation<br />
model that linked productivity and quality<br />
variables with accounting systems. Ton and Raman<br />
(2005) worked with a book retailer that was concerned<br />
about what it called “phantom stock outs,” instances<br />
in which a book was in a store but could not be found<br />
in response to a customer request. They tabulated data<br />
on instances <strong>of</strong> phantom stock outs and used <strong>the</strong> data<br />
to assess <strong>the</strong> level and causes <strong>of</strong> phantom stock outs<br />
as a precursorto designing countermeasures.<br />
In all <strong>the</strong> above examples, <strong>the</strong> data are about companies,<br />
gleaned from public or internal sources. Ulrich<br />
and Pearson (1998) sought to understand product<br />
design issues by directly examining products. Using<br />
an approach <strong>the</strong>y called “product archaeology,” <strong>the</strong>y<br />
took apart 20 c<strong>of</strong>fee makers, estimated manufacturing<br />
cost using techniques from Design for Manufacturability,<br />
and <strong>the</strong>n correlated cost with attributes <strong>of</strong> <strong>the</strong><br />
product’s design to understand how design attributes<br />
influence cost.<br />
As well as directly observing products, a researcher<br />
can directly observe processes. MacDuffie (1997) spent<br />
one week each at a GM, Ford, and Honda plant, documenting<br />
and comparing <strong>the</strong>ir approaches to solving<br />
waterleaks, paint defects, and electrical defects.<br />
Unlike o<strong>the</strong>r examples I have cited, his results were<br />
comprised <strong>of</strong> qualitative descriptions <strong>of</strong> <strong>the</strong> processes<br />
used in each <strong>of</strong> <strong>the</strong> plants, but <strong>the</strong>y are none<strong>the</strong>less<br />
interesting forthat fact. In a similarfashion, Tucker<br />
(2004, p. 4), “a management researcher with a background<br />
in quality engineering in manufacturing settings,<br />
spent 239 hours shadowing 26 different nurses<br />
at nine hospitals and recording detailed information<br />
about <strong>the</strong>irwork activities” to betterunderstand how<br />
<strong>the</strong>y dealt with operational failures.<br />
Experimentation is a standard tool <strong>of</strong> empirical<br />
research, a tool that has also proven useful to operations<br />
management researchers. Experiments can be<br />
conducted in a laboratory type setting or in a company.<br />
One example <strong>of</strong> laboratory experimentation<br />
already mentioned is <strong>the</strong> beer game (Sterman 1989),<br />
which asks students to make supply decisions in <strong>the</strong>
Fisher: <strong>Streng<strong>the</strong>ning</strong> <strong>the</strong> <strong>Empirical</strong> <strong>Base</strong> <strong>of</strong> <strong>Operations</strong> <strong>Management</strong><br />
Manufacturing & Service <strong>Operations</strong> <strong>Management</strong> 9(4), pp. 368–382, © 2007 INFORMS 379<br />
context <strong>of</strong> a mythical beersupply chain. Many o<strong>the</strong>r<br />
studies have placed students in supply chain-related<br />
decision contexts to understand <strong>the</strong> human side <strong>of</strong><br />
supply chain decision making. Schweitzerand Cachon<br />
(2000) provide a good example. Students were asked<br />
to make various types <strong>of</strong> newsvendor decisions to<br />
understand how and why <strong>the</strong>ir decisions differed<br />
from those recommended by <strong>the</strong> standard newsvendorformula.<br />
It is harder to convince a company to use itself as<br />
a laboratory, and it is <strong>the</strong>refore noteworthy that some<br />
researchers have succeeded in doing this. The application<br />
<strong>of</strong> a model and algorithm within a company is a<br />
kind <strong>of</strong> experiment, and I would regard <strong>the</strong> algorithm<br />
application examples in Bartholdi et al. (1983) and<br />
Bowman (1963) as examples <strong>of</strong> experiments within<br />
organizations. Burchill and Fine (1997) persuaded a<br />
number<strong>of</strong> companies to test a process <strong>the</strong> authors had<br />
developed forconcept engineering and to compare<br />
results using <strong>the</strong> new process versus <strong>the</strong>ir standard<br />
process. Lapre and Van Wassenhove (2001) worked<br />
with N. V. Bekaert, S.A. to conduct experiments using<br />
one <strong>of</strong> its production lines to learn <strong>the</strong> impact on<br />
productivity <strong>of</strong> various production parameters. The<br />
results <strong>of</strong> <strong>the</strong>se experiments are credited with enabling<br />
huge production improvements.<br />
8. Approaches to Conducting<br />
<strong>Empirical</strong> Research: Navigating<br />
<strong>the</strong> Matrix<br />
I have found two strategies to be particularly effective<br />
in conducting empirical research. Both involve<br />
conducting various types <strong>of</strong> empirical research corresponding<br />
to different cells <strong>of</strong> <strong>the</strong> matrix (Figure 1).<br />
I have always found it useful when contemplating<br />
research on a topic to start in <strong>the</strong> lower-right cell with<br />
discussions with one ormore companies. These discussions<br />
enable a deeperunderstanding <strong>of</strong> an issue so<br />
that subsequent research can be guided by better questions.<br />
It may make sense to write a case on one <strong>of</strong> <strong>the</strong><br />
companies with which one is interacting, which is a<br />
useful way to fur<strong>the</strong>r deepen one’s understanding <strong>of</strong><br />
issues.<br />
Sometimes, one will find a single company that<br />
has a well-defined problem that can be modeled and<br />
analyzed, so we start in <strong>the</strong> lower-right unstructured<br />
descriptive cell and move to <strong>the</strong> upper-left structured<br />
Figure 4 Navigating Matrix Cells—Approach 1<br />
Interaction with <strong>the</strong> world<br />
Highly<br />
structured:<br />
Data and<br />
algorithms<br />
Less structured:<br />
Interviews and<br />
observations<br />
Goal <strong>of</strong> <strong>the</strong> research<br />
Prescriptive Descriptive<br />
Engineering<br />
S<strong>of</strong>tware implementation<br />
<strong>of</strong> algorithm deployed in<br />
a company and run daily<br />
Principles<br />
Ohno sees U.S.<br />
supermarket and<br />
invents Toyota<br />
Production System<br />
<strong>Operations</strong> management<br />
econometrics<br />
Statistical analysis <strong>of</strong><br />
large data sets to<br />
discover drivers <strong>of</strong><br />
success in operations<br />
Start here<br />
Case studies<br />
Interview and observe<br />
managers<br />
Research cases<br />
prescriptive cell, following <strong>the</strong> path shown in Figure<br />
4. Many traditional projects that apply operations<br />
research/management science to a particular problem<br />
follow this path.<br />
Ano<strong>the</strong>r approach is illustrated in Figure 5. We start<br />
by interacting with a group <strong>of</strong> companies to form<br />
some hypo<strong>the</strong>ses, <strong>the</strong>n assemble a data set from an<br />
expanded set <strong>of</strong> companies to test <strong>the</strong> hypo<strong>the</strong>ses,<br />
and finally use <strong>the</strong> validated hypo<strong>the</strong>ses to design an<br />
agenda forimprovement.<br />
The research reported in Fisher et al. (1999) and<br />
Ramdas et al. (2003) followed this path for<strong>the</strong> issue<br />
<strong>of</strong> <strong>the</strong> best way to share parts across various finished<br />
products. We started in <strong>the</strong> lower-right cell by engaging<br />
in discussions with executives at several auto companies<br />
to understand how <strong>the</strong>y thought about this<br />
issue. We learned that <strong>the</strong>y viewed designing cars to<br />
maximize shared components as an important tool<br />
Figure 5 Navigating Matrix Cells—Approach 2<br />
Interaction with <strong>the</strong> world<br />
Highly<br />
structured:<br />
Data and<br />
algorithms<br />
Less structured:<br />
Interviews and<br />
observations<br />
Prescriptive<br />
Goal <strong>of</strong> <strong>the</strong> research<br />
Descriptive<br />
Engineering<br />
<strong>Operations</strong> management<br />
econometrics<br />
S<strong>of</strong>tware implementation Statistical analysis <strong>of</strong><br />
<strong>of</strong> algorithm deployed in large data sets to<br />
a company and run daily<br />
discover drivers <strong>of</strong><br />
success in operations<br />
Optimization<br />
Validation<br />
Hypo<strong>the</strong>ses Start here<br />
Principles<br />
Case studies<br />
Ohno sees U.S. Interview and observe<br />
supermarket and<br />
managers<br />
invents Toyota<br />
Production System<br />
Research cases
Fisher: <strong>Streng<strong>the</strong>ning</strong> <strong>the</strong> <strong>Empirical</strong> <strong>Base</strong> <strong>of</strong> <strong>Operations</strong> <strong>Management</strong><br />
380 Manufacturing & Service <strong>Operations</strong> <strong>Management</strong> 9(4), pp. 368–382, © 2007 INFORMS<br />
for providing maximum variety to customers while<br />
retaining less variety in plants. We also learned that<br />
many functional components could be rank ordered<br />
on a single quality metric. Brakes are an example,<br />
where <strong>the</strong> metric is stopping power. One could support<br />
an entire product line with a single brake type if<br />
it had <strong>the</strong> powerto stop <strong>the</strong> heaviest carin <strong>the</strong> line<br />
within a requisite distance, but <strong>the</strong> per unit production<br />
cost <strong>of</strong> this brake would be inordinately high for<br />
smaller cars. Conversely, you could minimize per unit<br />
production costs by having a unique brake for each<br />
car, but this would result in high plant complexity<br />
costs.<br />
We <strong>the</strong>n sought a principle that would guide our<br />
subsequent research and showed, for a stylized version<br />
<strong>of</strong> <strong>the</strong> problem, that <strong>the</strong> number <strong>of</strong> brakes that<br />
minimizes perunit production costs and <strong>the</strong> cost <strong>of</strong><br />
plant complexity from a greater number <strong>of</strong> brakes<br />
could be found via a model that resembled <strong>the</strong> economic<br />
order quantity model. From this we hypo<strong>the</strong>sized<br />
that <strong>the</strong> number<strong>of</strong> brakes an auto company<br />
would create to support a given product line would<br />
depend on factors such as <strong>the</strong> number <strong>of</strong> cars in <strong>the</strong><br />
line, <strong>the</strong> variance in weight <strong>of</strong> those cars, and <strong>the</strong>ir<br />
production volumes. We <strong>the</strong>n moved to <strong>the</strong> upperright<br />
cell to verify <strong>the</strong>se hypo<strong>the</strong>ses by analyzing a<br />
database assembled fora large number<strong>of</strong> auto companies<br />
using public data on auto specifications and<br />
production volumes and data provided as a service<br />
to salvage yards that showed commonality <strong>of</strong> brakes<br />
across cars. Finally, we moved to <strong>the</strong> upper-left cell by<br />
formulating and analyzing a model for determining<br />
<strong>the</strong> optimal number and type <strong>of</strong> brakes to support a<br />
defined product line.<br />
The approaches to research outlined here might be<br />
contrasted with a more common one <strong>of</strong> reading a<br />
paperin a journal and identifying a variant <strong>of</strong> it to<br />
be analyzed. While much good research falls in this<br />
category—and it’s fine to have this as part <strong>of</strong> a portfolio,<br />
if that’s all we do—we would be at risk, in<br />
von Neuman’s words, to “separate into a multitude <strong>of</strong><br />
insignificant branches.”<br />
9. Conclusions and Some Suggested<br />
Action Steps<br />
I have suggested that <strong>the</strong> field <strong>of</strong> operations management<br />
can benefit from streng<strong>the</strong>ning its empirical<br />
dimension. As evidence, I have <strong>of</strong>fered <strong>the</strong> examples<br />
<strong>of</strong> physics, medicine, and finance, all <strong>of</strong> which have a<br />
strong empirical tradition and are prospering. Moreover,<br />
<strong>the</strong>se fields provide role models for how empirical<br />
research should be conducted, which I have<br />
attempted to summarize in this paper.<br />
Some advantages <strong>of</strong> a strong empirical component<br />
to our research include <strong>the</strong> following:<br />
1. Identifying and verifying important phenomena<br />
2. Identifying and characterizing important questions<br />
on which we can do useful research<br />
3. Validating models and assumptions that we have<br />
made<br />
4. Establishing <strong>the</strong> relevance <strong>of</strong> our research by<br />
demonstrating how <strong>the</strong> research outputs apply to<br />
practice.<br />
If you agree with <strong>the</strong>se assertions, <strong>the</strong>n a natural<br />
question is what action steps should be taken. We can<br />
separate actions into those to be taken by individuals,<br />
academic departments, and pr<strong>of</strong>essional societies.<br />
As an individual, you could, I hope, considerei<strong>the</strong>r<br />
adding an empirical component to your own research<br />
portfolio orcontinuing yourempirical research if<br />
you already do it. Academic departments can considerdevoting<br />
some <strong>of</strong> <strong>the</strong>irhiring slots to faculty<br />
doing empirical research, introducing courses on<br />
empirical research into <strong>the</strong>ir PhD programs (<strong>the</strong> PhD<br />
course <strong>Empirical</strong> Research in <strong>Operations</strong> <strong>Management</strong><br />
designed and taught by Christian Terwiesch in <strong>the</strong><br />
<strong>Operations</strong> and Information <strong>Management</strong> Department<br />
at <strong>the</strong> Wharton School is one example) and giving<br />
PhD students a clinical experience via working on<br />
research projects within companies. A more ambitious<br />
goal would be to create more institutionalized<br />
opportunities for a clinical experience that might constitute<br />
a “teaching hospital” foroperations management.<br />
Some interesting initiatives in this direction<br />
are <strong>the</strong> internships provided within <strong>the</strong> MIT Leaders<br />
for Manufacturing program and <strong>the</strong> University<br />
<strong>of</strong> Michigan Manufacturing Applications Project program,<br />
in which faculty work with students on field<br />
projects with companies.<br />
There is also a huge leadership opportunity for pr<strong>of</strong>essional<br />
societies such as <strong>the</strong> Institute for<strong>Operations</strong><br />
Research and <strong>the</strong> <strong>Management</strong> Sciences (INFORMS)<br />
and <strong>the</strong> Production and <strong>Operations</strong> <strong>Management</strong> Society<br />
(POMS). Clearly, publication <strong>of</strong> empirical research
Fisher: <strong>Streng<strong>the</strong>ning</strong> <strong>the</strong> <strong>Empirical</strong> <strong>Base</strong> <strong>of</strong> <strong>Operations</strong> <strong>Management</strong><br />
Manufacturing & Service <strong>Operations</strong> <strong>Management</strong> 9(4), pp. 368–382, © 2007 INFORMS 381<br />
by journals such as Manufacturing & Service <strong>Operations</strong><br />
<strong>Management</strong> and Production and <strong>Operations</strong> <strong>Management</strong><br />
is important. This special issue <strong>of</strong> M&SOM on<br />
empirical methods in operations management, which<br />
is guest edited by Aleda Roth, is clearly a positive step<br />
in this direction. Gupta et al. (2006) also report encouraging<br />
evidence that publication <strong>of</strong> empirical research<br />
is increasing. I hope <strong>the</strong> journals will take <strong>the</strong> same<br />
broad view <strong>of</strong> empirical research <strong>of</strong>fered in this article<br />
and consider publishing interesting cases. Increased<br />
publication <strong>of</strong> broader empirical research will both<br />
require and encourage forging standards <strong>of</strong> what is<br />
good empirical research.<br />
Pr<strong>of</strong>essional societies could also sponsor industryacademic<br />
conferences to provide a context for academics<br />
to interact with practicing managers to learn<br />
<strong>the</strong> issues <strong>the</strong>y face and identify meaningful research<br />
topics. An example is <strong>the</strong> conference “Improving<br />
Supply Chain Synchronization and Strategy through<br />
Industry-Academia Collaboration” organized by <strong>the</strong><br />
POMS Supply Chain College and held at <strong>the</strong> University<br />
<strong>of</strong> Chicago on May 3, 2005.<br />
Finally, those who have conducted empirical research<br />
know that access to data is a challenge. Finance<br />
research into capital markets starting in <strong>the</strong> 1960s was<br />
greatly facilitated by <strong>the</strong> CenterforResearch in Security<br />
Prices at <strong>the</strong> University <strong>of</strong> Chicago; it provided<br />
a rich data set on security prices and related variables.<br />
It would be wonderful if a pr<strong>of</strong>essional society<br />
would considerwhat data would be useful for<br />
researchers in operations management and <strong>the</strong>n take<br />
on <strong>the</strong> task <strong>of</strong> maintaining those databases. Alternatively,<br />
one <strong>of</strong> <strong>the</strong> societies might act as a clearing house<br />
to enable individual researchers to make <strong>the</strong>ir data<br />
publicly available to o<strong>the</strong>rs. Happily, data sharing is<br />
starting to happen. Willems (2007) provides a data<br />
set that describes 38 real-world multiechelon supply<br />
chains that is publicly available through Manufacturing<br />
& Service <strong>Operations</strong> <strong>Management</strong> at <strong>the</strong> journal’s<br />
website (http://msom.pubs.informs.org).<br />
I sincerely believe that <strong>the</strong> pursuit <strong>of</strong> <strong>the</strong>se activities<br />
will have a tremendously vitalizing impact on our<br />
pr<strong>of</strong>ession, and I hope <strong>the</strong>se remarks will stimulate<br />
comments by o<strong>the</strong>rs on this issue.<br />
Acknowledgments<br />
The author is grateful to Garrett van Ryzin for suggesting<br />
this paperand forproviding encouragement and advice on<br />
drafts. The author also appreciates <strong>the</strong> helpful comments<br />
<strong>of</strong> Gérard Cachon, Nicole DeHoratius, Jan Fransoo, Vishal<br />
Gaur, Steve Graves, Michael Lapre, Hau Lee, John Paul Mac-<br />
Duffie, Kamalini Ramdas, Zeynep Ton, Karl Ulrich, Chris<br />
Voss, Luk van Wassenhove, and two anonymous reviewers.<br />
This paperis based on three talks <strong>the</strong> authorhas given on<br />
this subject: a talk in <strong>the</strong> MSOM Fellows Award session <strong>of</strong><br />
<strong>the</strong> 2002 INFORMS San Jose Meeting and plenary talks at<br />
<strong>the</strong> 2005 POMS Chicago Meeting and <strong>the</strong> 2006 INFORMS<br />
Hong Kong Meeting.<br />
References<br />
Bartholdi, J., L. Platzman, R. Collins, W. Warden. 1983. A minimal<br />
technology routing system for meals on wheels. Interfaces 13<br />
1–8.<br />
Bertrand, J. W. M., J. C. Fransoo. 2002. <strong>Operations</strong> management<br />
research methodologies using quantitative modeling. Internat.<br />
J. Oper. Production <strong>Management</strong> 22(2) 241–264.<br />
Bertrand, J. W. M., J. C. Fransoo. 2006. Modeling and simulation.<br />
C. Karlsson, ed. Researching <strong>Operations</strong> <strong>Management</strong>. Forthcoming.<br />
Bowman, E. 1963. Consistency and optimality in managerial decision<br />
making. <strong>Management</strong> Sci. 9(2) 310–321.<br />
Burchill, G., C. H. Fine. 1997. Time versus market orientation in<br />
product concept development: <strong>Empirical</strong>ly-based <strong>the</strong>ory generation.<br />
<strong>Management</strong> Sci. 43(4) 465–478.<br />
Cachon, G., M. Lariviere. 2001. Contracting to assure supply: How<br />
to share demand forecasts in a supply chain. <strong>Management</strong> Sci.<br />
47(5) 629–646.<br />
Cachon, G., M. Olivares. 2007. Competing retailers and inventory<br />
performance: An empirical investigation <strong>of</strong> U.S. automobile<br />
dealerships. Working paper, Wharton <strong>Operations</strong> and Information<br />
<strong>Management</strong>, Philadelphia, PA.<br />
Cachon, G., T. Randall, G. M. Schmidt. 2007. In search <strong>of</strong> <strong>the</strong><br />
bullwhip effect. Manufacturing Service Oper. <strong>Management</strong> 9(4)<br />
457–479.<br />
Clark, K. 1996. Competing through manufacturing and <strong>the</strong> new<br />
manufacturing paradigm: Is manufacturing strategy passé?<br />
Production Oper. <strong>Management</strong> 5(1) 42–58.<br />
Clark, K., T. Fujimoto. 1991. Product Development Performance. Harvard<br />
Business School Press, Cambridge, MA.<br />
Croson, R., K. Donohue. 2002. Experimental economics and supply<br />
chain management. Interfaces 32(5) 74–82.<br />
Datar, S., C. Jordan, S. Kekre, S. Rajiv, K. Srinivasan. 1997. New<br />
product development structures and time to market. <strong>Management</strong><br />
Sci. 43(4) 452–464.<br />
DeHoratius, N., A. Raman. 2004. Inventory record inaccuracy: An<br />
empirical analysis. Working paper, Harvard Business School,<br />
Cambridge, MA.<br />
DeHoratius, N., A. Raman. 2007. Store manager incentive design<br />
and retail performance: An exploratory investigation. Manufacturing<br />
Service Oper. <strong>Management</strong> 9(4) 518–534.<br />
Eisenhardt, K. M. 1989. Building <strong>the</strong>ories from case study research.<br />
Acad. <strong>Management</strong> J. 14(4) 532–550.<br />
Ferdows, K., A. DeMeyer. 1990. Lasting improvements in manufacturing<br />
performance: In search <strong>of</strong> a new <strong>the</strong>ory. J. Oper.<br />
<strong>Management</strong> 9(2) 168–184.
Fisher: <strong>Streng<strong>the</strong>ning</strong> <strong>the</strong> <strong>Empirical</strong> <strong>Base</strong> <strong>of</strong> <strong>Operations</strong> <strong>Management</strong><br />
382 Manufacturing & Service <strong>Operations</strong> <strong>Management</strong> 9(4), pp. 368–382, © 2007 INFORMS<br />
Fisher, M. 1991. Opportunities for operations research in <strong>the</strong> new<br />
manufacturing. Proc. Twelfth IFORMS Internat. Conf. Oper. Res.,<br />
Pergamon Press, A<strong>the</strong>ns, Greece, 579–588.<br />
Fisher, M. 1995. Vehicle routing. Handbooks in OR & MS 8 1–33.<br />
Fisher, M., K. Ramdas, K. Ulrich. 1999. Component sharing in <strong>the</strong><br />
management <strong>of</strong> product variety. <strong>Management</strong> Sci. 45(3) 297–315.<br />
Fisher, M. L. 1994. Optimal solution <strong>of</strong> vehicle routing problems<br />
using minimum K-trees. Oper. Res. 42(4) 626–642.<br />
Gaur, V., M. Fisher, A. Raman. 2005. An econometric analysis <strong>of</strong><br />
inventory turnover performance in retail services. <strong>Management</strong><br />
Sci. 51(2) 181–194.<br />
Gupta, S., R. Verma, L. Victorino. 2006. <strong>Empirical</strong> research published<br />
in Production and <strong>Operations</strong> <strong>Management</strong> (1992–2005):<br />
Trends and future research directions. Production Oper. <strong>Management</strong><br />
15(3) 432–448.<br />
Hayes, R. H., G. P. Pisano. 1996. Manufacturing strategy: At <strong>the</strong><br />
intersection <strong>of</strong> two paradigm shifts. Production Oper. <strong>Management</strong><br />
5(1) 25–41.<br />
Hendricks, K. B., V. R. Singhal. 1997. Delays in new product introductions<br />
and <strong>the</strong> market value <strong>of</strong> <strong>the</strong> firm: The consequences<br />
<strong>of</strong> being late to <strong>the</strong> market. <strong>Management</strong> Sci. 43(4) 422–436.<br />
Hendricks, K. B., V. R. Singhal. 2003. The effect <strong>of</strong> supply<br />
chain glitches on shareholder wealth. J. Oper. <strong>Management</strong> 21<br />
501–522.<br />
Holt, C., F. Modigliani, H. Simon. 1955. A lineardecision rule for<br />
production and employment scheduling. <strong>Management</strong> Sci. 2(1)<br />
1–30.<br />
Jaikumar, R. 1986. Westinghouse steam turbine generator diagnostic<br />
system. HBS Case 9-686-006, Harvard Business School,<br />
Boston, MA.<br />
Jordan, W., S. Graves. 1995. Principles on <strong>the</strong> benefits <strong>of</strong> manufacturing<br />
process flexibility. <strong>Management</strong> Sci. 41(4) 577–594.<br />
Keys, P. 1991. Operational Research and Systems: The Systemic Nature<br />
<strong>of</strong> Operational Research. Plenum Press, New York.<br />
Khanna, T., M. Iansiti. 1997. Firm asymmetries and sequential<br />
R and D. <strong>Management</strong> Sci. 43(4) 405–421.<br />
Krafcik, J. 1988. Triumph <strong>of</strong> <strong>the</strong> lean production system. Sloan <strong>Management</strong><br />
Rev. 30(1) 41–52.<br />
Lapre, M. A., G. D. Scudder. 2004. Performance improvement paths<br />
in <strong>the</strong> U.S. airline industry: Linking trade-<strong>of</strong>fs to asset frontiers.<br />
Production Oper. <strong>Management</strong> 13(2) 123–134.<br />
Lapre, M. A., L. N. van Wassenhove. 2001. Creating and transferring<br />
knowledge for productivity improvement in factories.<br />
<strong>Management</strong> Sci. 47(10) 1311–1325.<br />
Lee, H. L., V. Padmanabhan, S. Wang. 1997. Information distortion<br />
in a supply chain: The bullwhip effect. <strong>Management</strong> Sci. 43(4)<br />
546–559.<br />
MacDuffie, J. P. 1991. Beyond mass production: Flexible production<br />
systems and manufacturing performance in <strong>the</strong> world auto<br />
industry. Unpublished doctoral dissertation, MIT, Cambridge,<br />
MA.<br />
MacDuffie, J. P. 1997. The road to root cause: Shop-floor problemsolving<br />
at three auto assembly plans. <strong>Management</strong> Sci. 43(4)<br />
479–502.<br />
MacDuffie, J. P., K. Sethuraman, M. Fisher. 1996. Product variety<br />
and manufacturing performance: Evidence from <strong>the</strong> international<br />
automotive assembly plant study. <strong>Management</strong> Sci. 42(2)<br />
350–369.<br />
Mukherjee, A., M. Lapre, L. V. Wassenhove. 1998. Knowledge<br />
driven quality improvement. <strong>Management</strong> Sci. 44(11) S35–S49.<br />
Porter, M. E. 1996. What is strategy? Harvard Bus. Rev. 76(6) 61–78.<br />
Ramdas, K., M. Fisher, K. Ulrich. 2003. Managing variety for assembled<br />
products: Modeling component systems sharing. Manufacturing<br />
Service Oper. <strong>Management</strong> 5(2) 142–156.<br />
Randall, T., K. Ulrich. 2001. Product variety, supply chain structure,<br />
and firm performance: Analysis <strong>of</strong> <strong>the</strong> U.S. bicycle industry.<br />
<strong>Management</strong> Sci. 47(12) 1588–1604.<br />
Ross, S. M. 1971. Quality control under Markovian determination.<br />
<strong>Management</strong> Sci. 17 587–596.<br />
Schweitzer, M., G. P. Cachon. 2000. Decision bias in <strong>the</strong> newsvendor<br />
problem with a known demand distribution: Experimental<br />
evidence. <strong>Management</strong> Sci. 46(3) 404–420.<br />
Sterman, J., N. P. Repenning, F. K<strong>of</strong>man. 1997. Unanticipated side<br />
effects <strong>of</strong> successful quality programs: Exploring a paradox <strong>of</strong><br />
organizational improvement. <strong>Management</strong> Sci. 43(4) 503–521.<br />
Sterman, J. D. 1989. Modeling managerial behavior: Misperceptions<br />
<strong>of</strong> feedback in a dynamic decision making experiment.<br />
<strong>Management</strong> Sci. 35(3) 321–339.<br />
Terwiesch, C., Z. J. Ren, T. H. Ho, M. A. Cohen. 2005. An empirical<br />
analysis <strong>of</strong> forecast sharing in <strong>the</strong> semiconductor equipment<br />
supply chain. <strong>Management</strong> Sci. 51(2) 208–220.<br />
Ton, Z., A. Raman. 2005. The effect <strong>of</strong> product variety and inventory<br />
levels on retail store operations: A longitudinal study.<br />
Working paper, Harvard Business School, Cambridge, MA.<br />
Tucker, A. 2004. The impact <strong>of</strong> operational failures on hospital<br />
nurses and <strong>the</strong>ir patients. J. Oper. <strong>Management</strong> 22 151–169.<br />
Ulrich, K. T., S. Pearson. 1998. Assessing <strong>the</strong> importance <strong>of</strong> design<br />
through product archaeology. <strong>Management</strong> Sci. 44(3) 352–369.<br />
von Neumann, J. 1956. The ma<strong>the</strong>matician. J. R. Newman, ed. The<br />
World <strong>of</strong> Ma<strong>the</strong>matics, Vol. 4. Simon and Schuster, New York,<br />
2053–2063.<br />
Voss, C. A. 2006. <strong>Empirical</strong> research in operations management—<br />
Is <strong>the</strong>re a case for case research? Presented at <strong>the</strong> Wharton<br />
Workshop on <strong>Empirical</strong> Research in <strong>Operations</strong> <strong>Management</strong>,<br />
September28–29, The Wharton School, Philadelphia, PA.<br />
Voss, C. A., N. Tsikriktsis, M. Frohlich. 2002. Case research in operations<br />
management. Internat. J. Oper. Production <strong>Management</strong><br />
22(2) 195–219.<br />
Wheelright, S., J. Weber. 1995. Massachusetts General Hospital:<br />
CABG Surgery (A). HBS Case 9-696-015, Harvard Business<br />
School, Boston, MA.<br />
Willems, S. 2007. Real-world multi-echelon inventory optimization<br />
supply chains used forinventory optimization. Manufacturing<br />
Service Oper. <strong>Management</strong>. Forthcoming.<br />
Yin, R. K. 1994. Case Study Research: Design and Methods, 2nd ed.<br />
Sage Publication, Thousand Oaks, CA.