Modernist-Cuisine-Vol.-1-Small

4
Before you get alarmed, look at the 95% confidence
interval (the error bars on the chart). This
span of numbers reflects the statistical power of
the study, which depends on its size, length, and
design. In the case of the Women’s Health Initiative
study, the 95% confidence interval spans the
range from 0.90–1.29that is, from a 10% reduction
in cancer risk to a 29% increase in risk. The
study lacked the statistical power to distinguish
which number in that range represents the true
change in risk. A less technical way to say this is
that the trial results show that fiber might reduce
the risk of cancer by 10%, might increase cancer
risk by up to 29%, or might affect cancer risk by
a percentage anywhere between those two values.
The 95% confidence interval means that, in
principle, if you ran the study 100 times, then
you’d expect that, 95 times out of 100, the result
would fall within that range, and five times out of
100, the result would be outside the range. As
a general rule of thumb, if the confidence interval
includes 1.0, then the best interpretation is that
the comparative risk of the two groups is not
significantly different.
Unfortunately, this crucial point often gets lost
in mass media reports on medical research. The
statistics are so complicated to use and interpret
that even doctors themselves often misunderstand
the clinical implications (or lack thereof) of
a published study. In a landmark but controversial
2005 paper titled “Why Most Published Research
Findings Are False,” epidemiologist John Ioannidis
of Tufts University School of Medicine presented
mathematical arguments that the statistical and
experimental practices commonly used in epidemiological
research often produce misleading results.
Ioannidis concluded that, despite the use of
statistical tests like the 95% confidence interval,
more than half of all studies are likely to yield
incorrect results because of subtle flaws in their
statistical approaches. “For many current scientific
fields,” he wrote, “claimed research findings may
often be simply accurate measures of the prevailing
bias” of researchers in the field.
Even if that is too pessimistic a view, any
epidemiologist would agree that gathering strong
evidence in support of a dietary system is very
difficult. You need a very large, long-term, randomized,
controlled clinical trial, followed by a careful
statistical analysis of the results that factors out all
potential confounding variables. The process
works best for very dramatic results, such as the
link between smoking and lung cancer, that cannot
plausibly be explained by confounding factors. The
exact figures vary with gender and age, but smokers
have roughly 10 times (1,000%) the risk of lung
cancer that nonsmokers do.
Vitamin deficiencies produce similarly dramatic
effects, so they were conclusively identified long
ago. But science has now found most of those
dramatic effects. What remains unknown are the
links between diet and chronic diseases that have
far more intricate webs of contributory risk factors
and much subtler effects. Much larger numbers of
people are needed in trials that aim to uncover any
causal relationships among these factors.
How large is large enough to assure researchers
that the results are not due to chance or bias?
To estimate the right size, scientists must factor
in the prevalence of the condition under study
and the rate of new cases, the duration of the
study, the compliance rate of the volunteers, the
magnitude of biases and confounders, the
number of variables under study, and myriad
other considerations that affect the study’s
statistical power. There are no simple rules of
thumb, except that the rarer the condition, and
the smaller the effect you are looking for, the
larger the trial that you need.
To achieve proper statistical power, studying
dietary risk factors for even a relatively common
disease like heart disease requires a large trial that
costs as much as $250 million. That is why such
experimental evidence is so sparse in nutritional
epidemiology. To work around the cost and
complexity of running a single big trial, investigators
often pool data from many different studies
and use meta-analysis to approximate one large
study. This approach is not nearly as reliable as
a single large, well-designed, randomized trial. But
these are sometimes the best results that science is
able to produce.
TYPES OF NUTRITIONAL EPIDEMIOLOGY STUDIES
Ecological study
Case-control study
Each year, as the weather warms, newspapers roll out warnings
about alleged carcinogens (cancer-causing compounds)
in grilled meat and fish. Much of the concern revolves around
a set of chemicals called heterocyclic amines (HCAs), which
are produced by the Maillard reactions that cause seared
meat to brown. People get concerned about HCAs because
the U.S. National Toxicology Program says these chemicals
are “reasonably anticipated” to be carcinogens in humans.
The International Agency for Research on Cancer concurs.
When lab rats are fed high doses of these compounds, the
rats get more cancer than usual.
But does that mean the chemicals also cause cancer in
humans? Not necessarily. Many reports document that what
is true for rats or mice is not necessarily true for humans. And
a recent, very large prospective study that followed more than
120,000 women for eight years found no association between
breast cancer and red meat consumption or the way the red
meat was cooked. It is thus premature to declare that HCAs
are dangerous.
Several lines of evidence suggest that humans or our
ancestors adapted to eat cooked food, whereas lab rats and
their ancestors did not (see Origins of Cooking, page 6). It’s
Cohort, or
prospective, study
Kind observation observation observation intervention
Design
Strengths
Weaknesses
compares the nutrition and
health status of groups of
people at a particular time
good when data on
individuals are unavailable
or when differences between
individuals are small
results may not apply to
individuals; publication bias;
confounding effects
Weakest
C ONTR O V E RSIE S
Is Grilled Meat Bad for You?
compares people with
a disease (the cases)
to people without the
disease (the controls)
relatively quick and
inexpensive
sampling error; selection
bias; publication bias;
confounding effects;
observation bias; recall bias
tracks group members’
lifestyle and health status
over time, then looks for
correlations between diet
and disease
can eliminate recall bias
sampling error; selection
bias; publication bias;
confounding effects;
observation bias
Randomized clinical trial
assigns volunteers randomly to
an intervention group (that eats
a specified diet, for example) or
a control group (that carries on
as usual), then tracks subjects’
health
can reveal evidence of a causal
relationship between the
intervention and the outcome
sampling error; selection bias;
publication bias; confounding
effects; observation bias
Strongest
quite possible that we tolerate Maillard reaction products
better than rodents do—and even that some of the chemicals
are beneficial. Animal studies show, for example, that Maillard
products act as antioxidants and suppress the bacteria
responsible for peptic ulcers. And people seem to tolerate
breakfast cereal, crusty bread, and potato chips just fine,
even though those ubiquitous foods all get their toasted,
golden hue from Maillard reaction products such as acrylamide—which
also is “anticipated” to act as a carcinogen.
220 VOLUME 1 · HISTORY AND FUNDAMENTALS
FOOD AND HEALTH 221