Learning Data Mining with Python

Recommending Movies Using Affinity Analysis
However, it can be applied to many processes:
• Fraud detection
• Customer segmentation
• Software optimization
• Product recommendations
Affinity analysis is usually much more exploratory than classification. We often
don't have the complete dataset we expect for many classification tasks. For instance,
in movie recommendation, we have reviews from different people on different
movies. However, it is unlikely we have each reviewer review all of the movies in
our dataset. This leaves an important and difficult question in affinity analysis. If a
reviewer hasn't reviewed a movie, is that an indication that they aren't interested
in the movie (and therefore wouldn't recommend it) or simply that they haven't
reviewed it yet?
We won't answer that question in this chapter, but thinking about gaps in your
datasets can lead to questions like this. In turn, that can lead to answers that may
help improve the efficacy of your approach.
Algorithms for affinity analysis
We introduced a basic method for affinity analysis in Chapter 1, Getting Started with
Data Mining, which tested all of the possible rule combinations. We computed the
confidence and support for each rule, which in turn allowed us to rank them to find
the best rules.
However, this approach is not efficient. Our dataset in Chapter 1, Getting Started with
Data Mining, had just five items for sale. We could expect even a small store to have
hundreds of items for sale, while many online stores would have thousands (or
millions!). With a naive rule creation, such as our previous algorithm, the growth in
time needed to compute these rules increases exponentially. As we add more items,
the time it takes to compute all rules increases significantly faster. Specifically, the
total possible number of rules is 2n - 1. For our five-item dataset, there are 31 possible
rules. For 10 items, it is 1023. For just 100 items, the number has 30 digits. Even the
drastic increase in computing power couldn't possibly keep up with the increases in
the number of items stored online. Therefore, we need algorithms that work smarter,
as opposed to computers that work harder.
[ 62 ]