Learning Data Mining with Python

Chapter 4
We want to break out the preceding loop if we didn't find any new frequent itemsets
(and also to print a message to let us know what is going on):
if len(cur_frequent_itemsets) == 0:
print("Did not find any frequent itemsets of length {}".
format(k))
sys.stdout.flush()
break
We use sys.stdout.flush() to ensure that the printouts happen
while the code is still running. Sometimes, in large loops in particular
cells, the printouts will not happen until the code has completed. Flushing
the output in this way ensures that the printout happens when we want.
Don't do it too much though—the flush operation carries a computational
cost (as does printing) and this will slow down the program.
If we do find frequent itemsets, we print out a message to let us know the loop will
be running again. This algorithm can take a while to run, so it is helpful to know that
the code is still running while you wait for it to complete! Let's look at the code:
else:
print("I found {} frequent itemsets of length
{}".format(len(cur_frequent_itemsets), k))
sys.stdout.flush()
Finally, after the end of the loop, we are no longer interested in the first set of
itemsets anymore—these are itemsets of length one, which won't help us create
association rules – we need at least two items to create association rules. Let's
delete them:
del frequent_itemsets[1]
You can now run this code. It may take a few minutes, more if you have older
hardware. If you find you are having trouble running any of the code samples,
take a look at using an online cloud provider for additional speed. Details about
using the cloud to do the work are given in Appendix, Next Steps.
The preceding code returns 1,718 frequent itemsets of varying lengths. You'll notice
that the number of itemsets grows as the length increases before it shrinks. It grows
because of the increasing number of possible rules. After a while, the large number
of combinations no longer has the support necessary to be considered frequent.
This results in the number shrinking. This shrinking is the benefit of the Apriori
algorithm. If we search all possible itemsets (not just the supersets of frequent ones),
we would be searching thousands of times more itemsets to see if they are frequent.
[ 71 ]