Building Machine Learning Systems with Python - Richert, Coelho

Topic Modeling
Sparsity means that while you may have large matrices and vectors,
in principle, most of the values are zero (or so small that we can round
them to zero as a good approximation). Therefore, only a few things
are relevant at any given time.
Often problems that seem too big to solve are actually feasible because
the data is sparse. For example, even though one webpage can link to
any other webpage, the graph of links is actually very sparse as each
webpage will link to a very tiny fraction of all other webpages.
In the previous graph, we can see that about 150 documents have 5 topics, while
the majority deal with around 10 to 12 of them. No document talks about more
than 20 topics.
To a large extent, this is a function of the parameters used, namely the alpha
parameter. The exact meaning of alpha is a bit abstract, but bigger values for alpha
will result in more topics per document. Alpha needs to be positive, but is typically
very small; usually smaller than one. By default, gensim will set alpha equal to 1.0/
len (corpus), but you can set it yourself as follows:
>>> model = models.ldamodel.LdaModel(
corpus,
num_topics=100,
id2word=corpus.id2word,
alpha=1)
In this case, this is a larger alpha, which should lead to more topics per document.
We could also use a smaller value. As we can see in the combined histogram given
next, gensim behaves as we expected:
[ 78 ]