Learning Data Mining with Python

Clustering News Articles
In most of the previous chapters, we performed data mining knowing what we
were looking for. Our use of target classes allowed us to learn how our variables
model those targets during the training phase. This type of learning, where we have
targets to train against, is called supervised learning. In this chapter, we consider
what we do without those targets. This is unsupervised learning and is much more
of an exploratory task. Rather than wanting to classify with our model, the goal in
unsupervised learning is more about exploring the data to find insights.
In this chapter, we look at clustering news articles to find trends and patterns in
the data. We look at how we can extract data from different websites using a link
aggregation website to show a variety of news stories.
The key concepts covered in this chapter include:
• Obtaining text from arbitrary websites
• Using the reddit API to collect interesting news stories
• Cluster analysis for unsupervised data mining
• Extracting topics from documents
• Online learning for updating a model without retraining it
• Cluster ensembling to combine different models
Obtaining news articles
In this chapter, we will build a system that takes a live feed of news articles and
groups them together, where the groups have similar topics. You could run the
system over several weeks (or longer) to see how trends change over that time.
[ 211 ]