Learning Data Mining with Python

Chapter 4
Implementation
On the first iteration of Apriori, the newly discovered itemsets will have a length
of 2, as they will be supersets of the initial itemsets created in the first step. On the
second iteration (after applying the fourth step), the newly discovered itemsets will
have a length of 3. This allows us to quickly identify the newly discovered itemsets,
as needed in second step.
We can store our discovered frequent itemsets in a dictionary, where the key is the
length of the itemsets. This allows us to quickly access the itemsets of a given length,
and therefore the most recently discovered frequent itemsets, with the help of the
following code:
frequent_itemsets = {}
We also need to define the minimum support needed for an itemset to be considered
frequent. This value is chosen based on the dataset but feel free to try different
values. I recommend only changing it by 10 percent at a time though, as the time the
algorithm takes to run will be significantly different! Let's apply minimum support:
min_support = 50
To implement the first step of the Apriori algorithm, we create an itemset with each
movie individually and test if the itemset is frequent. We use frozenset, as they
allow us to perform set operations later on, and they can also be used as keys in our
counting dictionary (normal sets cannot). Let's look at the following code:
frequent_itemsets[1] = dict((frozenset((movie_id,)),
row["Favorable"])
for movie_id, row in num_favorable_
by_movie.iterrows()
if row["Favorable"] > min_support)
We implement the second and third steps together for efficiency by creating a
function that takes the newly discovered frequent itemsets, creates the supersets,
and then tests if they are frequent. First, we set up the function and the counting
dictionary:
from collections import defaultdict
def find_frequent_itemsets(favorable_reviews_by_users, k_1_itemsets,
min_support):
counts = defaultdict(int)
[ 69 ]