Building Machine Learning Systems with Python - Richert, Coelho

Classification II – Sentiment Analysis
The prior and evidence values are easily determined:
• is the prior probability of class without knowing about the data. This
quantity can be obtained by simply calculating the fraction of all training
data instances belonging to that particular class.
• is the evidence, or the probability of features and . This can be
retrieved by calculating the fraction of all training data instances having that
particular feature value.
• The tricky part is the calculation of the likelihood . It is the value
describing how likely it is to see feature values and if we know that the
class of the data instance is . To estimate this we need a bit more thinking.
Being naive
From the probability theory, we also know the following relationship:
This alone, however, does not help much, since we treat one difficult problem
(estimating ) with another one (estimating ).
However, if we naively assume that and are independent from each other,
simplifies to and we can write it as follows:
Putting everything together, we get this quite manageable formula:
The interesting thing is that although it is not theoretically correct to simply tweak
our assumptions when we are in the mood to do so, in this case it proves to work
astonishingly well in real-world applications.
[ 120 ]