Mathematics for Computer Science

“mcs” — 2017/3/3 — 11:21 — page 762 — #770
762
Chapter 18
Conditional Probability
so
Pr A j B D 9 10 D Pr A j C :
Also, since 0 is the only outcome in B \ C and 0 … A, we have
Pr A j B \ C D 0
So the right-hand side of (18.3) is 1.8, while the left-hand side is a probability
which can be at most 1—actually, it is 18/19.
18.6 Simpson’s Paradox
In 1973, a famous university was investigated for gender discrimination [6]. The
investigation was prompted by evidence that, at first glance, appeared definitive: in
1973, 44% of male applicants to the school’s graduate programs were accepted, but
only 35% of female applicants were admitted.
However, this data turned out to be completely misleading. Analysis of the individual
departments, showed not only that few showed significant evidence of
bias, but also that among the few departments that did show statistical irregularities,
most were slanted in favor of women. This suggests that if there was any sex
discrimination, then it was against men!
Given the discrepancy in these findings, it feels like someone must be doing bad
math—intentionally or otherwise. But the numbers are not actually inconsistent.
In fact, this statistical hiccup is common enough to merit its own name: Simpson’s
Paradox occurs when multiple small groups of data all exhibit a similar trend, but
that trend reverses when those groups are aggregated. To explain how this is possible,
let’s first clarify the problem by expressing both arguments in terms of conditional
probabilities. For simplicity, suppose that there are only two departments
EE and CS. Consider the experiment where we pick a random candidate. Define
the following events:
AWWD the candidate is admitted to his or her program of choice,
F EE WWD the candidate is a woman applying to the EE department,
F CS WWD the candidate is a woman applying to the CS department,
M EE WWD the candidate is a man applying to the EE department,
M CS WWD the candidate is a man applying to the CS department.