Nonparametric Bayesian Discrete Latent Variable Models for ...

More documents

Recommendations

Info

3 Dirichlet Process Mixture Models c1 c2 c3 c4 c5 c6 c7 c8 c9 c10 Figure 3.23: Example spike waveforms assigned to different clusters by the DPMFA model using the whole waveforms as inputs. The waveforms assigned to clusters 1, 2 and 3 have similar amplitude characteristics. Therefore, they were assumed to belong to one big cluster in manual clustering since they have similar amplitude characteristics. DPMFA can discover that they should belong to different clusters by using the waveform information. On the other hand, the waveforms assigned to the clusters c6 and c7 appear to be very similar. It is possible that due to the incremental updates, the sampler fails to merge these two components together which belong to the same cluster. This is also the case for the waveforms assigned to components c8 and c9. 64
3.5 Discussion 3.5 Discussion In this chapter, we have considered the DPM models for density estimation and clustering. The DP has been introduced by Ferguson (1973) and has been extensively used especially since the development of MCMC methods for inference. There are several different approaches to define the DP. In the beginning of this chapter, we have summarized some of these approaches to give an insight about the DP and its properties. The different ways of defining the same distribution has lead to several different inference algorithms for DPM models. We have outlined some of the MCMC algorithms developed for inference on the DPM models. The list of algorithms that we described is not exhaustive, but we believe that they give a good overview of the development of the techniques. We compared the DPMoG models with conjugate and conditionally conjugate base distributions. We showed that for the inference algorithm we used, mixing time of the samplers can be vastly improved for the conditionally conjugate model by integrating out one of the parameters while conditioning on the other. Using this improved sampling scheme, inference for the conditionally conjugate model is not always computationally more expensive than the conjugate one. The relative mixing performance depends on the data modeled. The modeling performance of the conditionally conjugate model for density estimation is found to be always better than the fully conjugate one, the difference being significant in higher dimensions. These results suggest that one does not have to resort to using the conjugate base distribution when it does not represent the prior beliefs. We have introduced the DPMFA model for modeling high dimensional data that is believed to have a low dimensional representation as mixture of Gaussians with constrained covariance matrices. We have demonstrated the modeling performance of the DPMFA on a challenging clustering problem. Although the resulting clustering is successful, the incremental sampling algorithm used is not feasible for practical use of the model on high dimensional data and should be improved. Difference between the DPM models and mixtures with finite but unknown number of components is that the DPM takes into account the possibility of a yet not observed data point to be coming from an unrepresented component through the concentration parameter of the DPM both during training and when the predictive probabilities are calculated. Although the RJMCMC or BDMCMC give a distribution over the possible number of components, once an iteration is complete and a number of components is chosen, it is only those components that are used to explain the new data. The comparison of the performance if these models remains to be an interesting question. 65
Page 1:
Nonparametric Bayesian Discrete Lat
Page 4 and 5:
Matrizen mit unendlich vielen Spalt
Page 7 and 8:
Contents Zusammenfassung iii Abstra
Page 9:
List of Algorithms 1 Gibbs sampling
Page 13 and 14:
Notation Matrices are capitalized a
Page 15:
Symbol Meaning IBP Z binary latent
Page 18 and 19:
1 Introduction belief in the prior.
Page 20 and 21:
2 Nonparametric Bayesian Analysis b
Page 22 and 23:
2 Nonparametric Bayesian Analysis s
Page 25 and 26:
3 Dirichlet Process Mixture Models
Page 27 and 28:
3.1 The Dirichlet Process the perfo
Page 29 and 30: α G o G θi x i N 3.1 The Dirichle
Page 31 and 32: 15 10 5 −0.5 0 0.5 2 1 G 0 0 −0
Page 33 and 34: increment process with the correspo
Page 35 and 36: α G o π k c i θk x i 8 N 3.1 The
Page 37 and 38: 3.1 The Dirichlet Process Eq. (3.21
Page 39 and 40: number of components, K number of c
Page 41 and 42: 3.2 MCMC Inference in Dirichlet Pro
Page 43 and 44: and Bush and MacEachern (1996). 3.2
Page 45 and 46: 3.2.2 Algorithms for non-Conjugate
Page 55 and 56: ∗ π ∗ π s 3.2 MCMC Inference
Page 57 and 58: model can be written in the form of
Page 59 and 60: −1 µ y Σy Σy D ξ normal R 3.3
Page 61 and 62: the log likelihood term is: where a
Page 63 and 64: 3.3 Empirical Study on the Choice o
Page 65 and 66: autocovariance coefficient 1 0.8 0.
Page 67 and 68: # of data points # of data points 5
Page 69 and 70: 3.4 Dirichlet Process Mixtures of F
Page 71 and 72: µ y Σy ξ R 0 ν w normal µ −1
Page 73 and 74: ch1 ch2 ch3 ch4 3.4 Dirichlet Proce
Page 75 and 76: 3.4 Dirichlet Process Mixtures of F
Page 77 and 78: # of components # of components # o
Page 79: ch 2 ch 3 ch 1 ch 2 ch 3 ch 4 3.4 D
Page 84 and 85: 4 Indian Buffet Process Models matr
Page 86 and 87: 4 Indian Buffet Process Models In t
Page 88 and 89: 4 Indian Buffet Process Models α z
Page 90 and 91: 4 Indian Buffet Process Models α
Page 92 and 93: 4 Indian Buffet Process Models The
Page 94 and 95: 4 Indian Buffet Process Models colu
Page 96 and 97: 4 Indian Buffet Process Models Pois
Page 98 and 99: 4 Indian Buffet Process Models z α
Page 100 and 101: 4 Indian Buffet Process Models For
Page 102 and 103: 4 Indian Buffet Process Models ciat
Page 104 and 105: 4 Indian Buffet Process Models rati
Page 106 and 107: 4 Indian Buffet Process Models repr
Page 108 and 109: 4 Indian Buffet Process Models samp
Page 110 and 111: 4 Indian Buffet Process Models feat
Page 112 and 113: 4 Indian Buffet Process Models Algo
Page 114 and 115: 4 Indian Buffet Process Models mixi
Page 116 and 117: 4 Indian Buffet Process Models Figu
Page 118 and 119: 4 Indian Buffet Process Models pres
Page 120 and 121: 4 Indian Buffet Process Models ε
Page 122 and 123: 4 Indian Buffet Process Models LL P
Page 124 and 125: 4 Indian Buffet Process Models P+ P
Page 126 and 127: 4 Indian Buffet Process Models tEBA
Page 128 and 129: 4 Indian Buffet Process Models dist
Page 130 and 131:
5 Conclusions has been defined as a
Page 132 and 133:
A Details of Derivations for the St
Page 135 and 136:
B Mathematical Appendix B.1 Dirichl
Page 137 and 138:
p3 α 1 0.5 0 0 0.5 0.4 0.3 0.2 0.1
Page 139 and 140:
Construction of A Process B.4 Equal
Page 141 and 142:
Bibliography D. Aldous. Exchangeabi
Page 143 and 144:
Bibliography T. S. Ferguson. Prior
Page 145 and 146:
Bibliography L. F. James and J. W.
Page 147 and 148:
Bibliography R. M. Neal. Probabilis
Page 149 and 150:
Bibliography Y. W. Teh, M. I. Jorda
show all

Nonparametric Bayesian Discrete Latent Variable Models for ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?