Learning Data Mining with Python

Social Media Insight Using
Naive Bayes
Text-based datasets contain a lot of information, whether they are books, historical
documents, social media, e-mail, or any of the other ways we communicate via
writing. Extracting features from text-based datasets and using them for classification
is a difficult problem. There are, however, some common patterns for text mining.
We look at disambiguating terms in social media using the Naive Bayes algorithm,
which is a powerful and surprisingly simple algorithm. Naive Bayes takes a few
shortcuts to properly compute the probabilities for classification, hence the term
naive in the name. It can also be extended to other types of datasets quite easily and
doesn't rely on numerical features. The model in this chapter is a baseline for text
mining studies, as the process can work reasonably well for a variety of datasets.
We will cover the following topics in this chapter:
• Downloading data from social network APIs
• Transformers for text
• Naive Bayes classifier
• Using JSON for saving and loading datasets
• The NLTK library for extracting features from text
• The F-measure for evaluation
[ 105 ]