Nonparametric Bayesian Discrete Latent Variable Models for ...

5 Conclusions
The analysis of real-world problems demands a principled way of summarizing the data
and representing the uncertainty in these summaries. Bayesian methods allow representing
prior belief about the system generating the data and provides uncertainties
about the predictions. The analysis of complex systems often requires flexible models
that accurately capture the dependencies in the data. Simple parametric models can
become inadequate for complex real-world problems. Nonparametric methods are powerful
tools that allow definition of flexible models, since they can be used to approximate
any of a wide class of distributions.
The Dirichlet process (DP) is one of the most prominent random probability distributions.
Although introduced in the 70s, its use have become popular only recently due to
the development of sophisticated inference algorithms. The Indian buffet process (IBP)
is a generalization of the related Chinese restaurant process which allows definition of
more powerful models. In this thesis, we have presented empirical analysis of models
using the DP and the IBP.
The DP is defined by two parameters, a base distribution and a concentration parameter.
The base distribution is a probability distribution, determining the mean
of the DP. The concentration parameter is a positive scalar that can be seen as the
strength of belief in the prior guess. Use of conjugate priors makes computations for
inference much easier for Bayesian models in general. However, conjugate priors may
fail to represent one’s prior belief. The trade off between the computational ease and
modeling performance is an important modeling question. The Dirichlet process mixtures
of Gaussians (DPMoG) model is one of the most widely used Dirichlet process
mixture (DPM) models. We have empirically addressed this question in the DPMoG
using conjugate and conditionally conjugate base distributions. We have compared the
modeling performance and the computational cost of the inference algorithms for both
prior specifications. We have empirically shown that it is possible to increase computational
efficiency by exploiting conditional conjugacy while obtaining better modeling
performance than the fully conjugate model.
DPM models can be seen as mixture models with infinitely many components. Although
the number of components are not bounded, there is an inherent clustering
property in DPM models. We formulated a nonparametric form of the mixtures of factor
analyzers (MFA) model using the DP. The MFA models the data as a mixture of
Gaussians with reduced parametrization. Hence, the Dirichlet process MFA (DPMFA)
model allows modeling high dimensional data efficiently. We have exploited the clustering
property of the DPMs for the task of spike sorting, that is, clustering of spike
waveforms that arise from different neurons in an extracellular recording.
The IBP, a distribution over sparse binary matrices with infinitely many columns,
113