Learning Data Mining with Python

Recommending Movies Using Affinity Analysis
In keeping with our rule of thumb of reading through the data as little as possible,
we iterate over the dataset once per call to this function. While this doesn't matter too
much in this implementation (our dataset is relatively small), it is a good practice to
get into for larger applications. We iterate over all of the users and their reviews:
for user, reviews in favorable_reviews_by_users.items():
Next, we go through each of the previously discovered itemsets and see if it is a
subset of the current set of reviews. If it is, this means that the user has reviewed
each movie in the itemset. Let's look at the code:
for itemset in k_1_itemsets:
if itemset.issubset(reviews):
We can then go through each individual movie that the user has reviewed that isn't
in the itemset, create a superset from it, and record in our counting dictionary that
we saw this particular itemset. Let's look at the code:
for other_reviewed_movie in reviews - itemset:
current_superset = itemset | frozenset((other_
reviewed_movie,))
counts[current_superset] += 1
We end our function by testing which of the candidate itemsets have enough support
to be considered frequent and return only those:
return dict([(itemset, frequency) for itemset, frequency in
counts.items() if frequency >= min_support])
To run our code, we now create a loop that iterates over the steps of the Apriori
algorithm, storing the new itemsets as we go. In this loop, k represents the length
of the soon-to-be discovered frequent itemsets, allowing us to access the previously
most discovered ones by looking in our frequent_itemsets dictionary using the
key k - 1. We create the frequent itemsets and store them in our dictionary by their
length. Let's look at the code:
for k in range(2, 20):
cur_frequent_itemsets =
find_frequent_itemsets(favorable_reviews_by_users,
frequent_itemsets[k-1],
min_support)
frequent_itemsets[k] = cur_frequent_itemsets
[ 70 ]