Learning Data Mining with Python

Chapter 10
Neural networks can also be trained in batch mode, where a group of samples are
given at once and the training is done in one step. Algorithms are faster in batch
mode, but use more memory.
In this same vein, we can slightly update the k-means centroids after a single or small
batch of samples. To do this, we apply a learning rate to the centroid movement in
the updating step of the k-means algorithm. Assuming that samples are randomly
chosen from the population, the centroids should tend to move towards the positions
they would have in the standard, offline, and k-means algorithm.
Online learning is related to streaming-based learning; however, there are some
important differences. Online learning is capable of reviewing older samples after they
have been used in the model, while a streaming-based machine learning algorithm
typically only gets one pass—that is, one opportunity to look at each sample.
Implementation
The scikit-learn package contains the MiniBatchKMeans algorithm, which allows
online learning. This class implements a partial_fit function, which takes a set of
samples and updates the model. In contrast, calling fit() will remove any previous
training and refit the model only on the new data.
MiniBatchKMeans follows the same clustering format as other algorithms in
scikit-learn, so creating and using it is much the same as other algorithms.
Therefore, we can create a matrix X by extracting features from our dataset using
TfIDFVectorizer, and then sample from this to incrementally update our model.
The code is as follows:
vec = TfidfVectorizer(max_df=0.4)
X = vec.fit_transform(documents)
We then import MiniBatchKMeans and create an instance of it:
from sklearn.cluster import MiniBatchKMeans
mbkm = MiniBatchKMeans(random_state=14, n_clusters=3)
Next, we will randomly sample from our X matrix to simulate data coming in from
an external source. Each time we get some data in, we update the model:
batch_size = 10
for iteration in range(int(X.shape[0] / batch_size)):
start = batch_size * iteration
end = batch_size * (iteration + 1)
mbkm.partial_fit(X[start:end])
[ 237 ]