Building Machine Learning Systems with Python - Richert, Coelho

Regression – Recommendations Improved
We can see, for example, that there were 80 transactions of which 1378, 1379l,
and 1380 were bought together. Of these, 57 also included 1269, so the estimated
conditional probability is 57/80 ≈ 71%. Compared to the fact that only 0.3% of all
transactions included 1269 , this gives us a lift of 255.
The need to have a decent number of transactions in these counts in order to be able
to make relatively solid inferences is why we must first select frequent itemsets.
If we were to generate rules from an infrequent itemset, the counts would be very
small; due to this, the relative values would be meaningless (or subject to very large
error bars).
Note that there are many more association rules that have been discovered from
this dataset: 1030 datasets are required to support at least 80 minimum baskets
and a minimum lift of 5. This is still a small dataset when compared to what is now
possible with the Web; when you perform millions of transactions, you can expect to
generate many thousands, even millions, of rules.
However, for each customer, only a few of them will be relevant at any given time,
and so each costumer only receives a small number of recommendations.
More advanced basket analysis
There are now other algorithms for basket analysis that run faster than Apriori. The
code we saw earlier was simple, and was good enough for us as we only had circa
100 thousand transactions. If you have had many millions, it might be worthwhile to
use a faster algorithm (although note that for most applications, learning association
rules can be run offline).
There are also methods to work with temporal information leading to rules that take
into account the order in which you have made your purchases. To take an extreme
example of why this may be useful, consider that someone buying supplies for a
large party may come back for trash bags. Therefore it may make sense to propose
trash bags on the first visit. However, it would not make sense to propose party
supplies to everyone who buys a trash bag.
You can find Python open source implementations (a new BSD license as scikit-learn)
of some of these in a package called pymining. This package was developed by
Barthelemy Dagenais and is available at https://github.com/bartdag/pymining.
[ 178 ]