From Algorithms to Z-Scores - matloff - University of California, Davis

17.2. THE CLASSIFICATION PROBLEM 337
not contain “NIH,” and make five claims would have the value
1
1 + e −(β0+β1+5β3)
17.2.3.1 The Logistic Model: Motivations
The logistic function itself,
1
1 + e −u
(17.10)
(17.11)
has values between 0 and 1, and is thus a candidate for modeling a probability. Also, it is monotonic
in u, making it further attractive, as in many classification problems we believe that mY ;X(t) should
be monotonic in the predictor variables.
But there are additional reasons to use the logit model, as it includes many common parametric
models for X. To see this, note that we can write, for vector-valued discrete X and t,
P (Y = 1|X = t) =
= P (Y = 1)P (X = t|Y = 1)
=
=
P (Y = 1 and X = t)
P (X = t)
P (X = t)
P (Y = 1)P (X = t|Y = 1)
P (Y = 1)P (X = t|Y = 1) + P (Y = 0)P (X = t|Y = 0)
1
1 +
(1−q)P (X=t|Y =0)
qP (X=t|Y =1)
(17.12)
(17.13)
(17.14)
(17.15)
where q = P (Y = 1) is the proportion of members of the population which have Y = 1. (Keep in
mind that this probability is unconditional!!!! In the patent example, for instance, if say q = 0.12,
then 12% of all patents in the patent population—without regard to words used, numbers of claims,
etc.—are publicly funded.)
If X is a continuous random vector, then the analog of (17.15) is
P (Y = 1|X = t) =
1
1 + (1−q)f X|Y =0(t)
qf X|Y =1(t)
(17.16)