1FfUrl0

Chapter 6
Similarly, we do this for the prior probabilities:
Accounting for arithmetic underflows
There is yet another roadblock. In reality, we work with probabilities much smaller
than the ones we have dealt with in the toy example. In reality, we also have more
than two features, which we multiply with each other. This will quickly lead to the
point where the accuracy provided by NumPy does not suffice anymore:
>>> import numpy as np
>>> np.set_printoptions(precision=20) # tell numpy to print out more
digits (default is 8)
>>> np.array([2.48E-324])
array([ 4.94065645841246544177e-324])
>>> np.array([2.47E-324])
array([ 0.])
So, how probable is it that we will ever hit a number like 2.47E-324? To answer this,
we just have to imagine a likelihood for the conditional probabilities of 0.0001 and
then multiply 65 of them together (meaning that we have 65 low probable feature
values) and you've been hit by the arithmetic underflow:
>>> x=0.00001
>>> x**64 # still fine
1e-320
>>> x**65 # ouch
0.0
A float in Python is typically implemented using double in C. To find out whether
it is the case for your platform, you can check it as follows:
>>> import sys
>>> sys.float_info
sys.float_info(max=1.7976931348623157e+308, max_exp=1024, max_10_
exp=308, min=2.2250738585072014e-308, min_exp=-1021, min_10_exp=-307,
dig=15, mant_dig=53, epsilon=2.220446049250313e-16, radix=2, rounds=1)
To mitigate this, you could switch to math libraries such as mpmath (http://code.
google.com/p/mpmath/) that allow arbitrary accuracy. However, they are not fast
enough to work as a NumPy replacement.
[ 125 ]