Nonparametric Bayesian Discrete Latent Variable Models for ...

More documents

Recommendations

Info

4 Indian Buffet Process Models tEBA 15 10 5 NLL 5 10 BTL 15 iEBA 15 10 5 NLL 5 10 BTL 15 tEBA 15 10 5 NLL 5 10 iEBA 15 Figure 4.17: The negative log likelihood of one model versus another. Each point corresponds to one paired comparison. Table 4.2: Predictive performance of different models: The baseline model that always predicts a probability of 0.5 (BASE), the Bradley-Terry-Luce model (BTL), the finite EBA model with 12 features (EBA12), finite EBA model with 15 features (EBA15), EBA model with tree structure (tEBA), the EBA model with IBP prior (iEBA), and for comparison the empirical probabilities (EMP). NLL: The negative log likelihood values on the mean predictive probabilities. IS: Information score in bits (the information gain compared to the model that always predicts 0.5). NLL and IS both express the same values in different scales. model NLL IS BASE 17.57 0 BTL 4.66 0.0795 EBA12 4.50 0.0806 EBA15 4.31 0.0817 tEBA 3.95 0.0839 iEBA 3.92 0.0841 EMP 2.89 0.0905 many features will be necessary beforehand this is a strong argument in the favor of using a non-parametric prior. Figure 4.17 shows a more fine-grained analysis of the BTL, tEBA and iEBA models. Each point corresponds to one paired comparison: the negative log likelihood of one model versus the negative log likelihood of another model. Out of 36 pairs BTL has a smaller log likelihood for 23 of the pairs when compared to tEBA and 26 when compared to iEBA. However, it can be seen that the bad performance of the BTL model on average is also due to the fact that it cannot capture the probabilities of some pairs at all. The iEBA and tEBA likelihoods are comparable although there are some pairs on which iEBA performs better than tEBA, and vice versa. 110
4.5.4 Conclusions 4.6 Discussion EBA is a choice model which has correspondences to several models in economics and psychology. The model assumes the choice probabilities to result from the non-shared features of the options. The usefulness of the EBA model has been hampered by the lack of a method that can accurately infer the unknown features of the alternatives. We have suggested to use an infinite latent feature matrix to represent the unknown features. We showed empirically that the infinite latent feature model (iEBA) can capture the latent structure in the choice data as well as the handcrafted model (tEBA). For data for which we have less prior information it might not be possible to handcraft a reasonable feature matrix. Different feature matrices can result in the same choice probabilities and therefore are not distinguished by the model. For example, features that are shared by all options do not affect the likelihood. Furthermore, only the ratio of the weights affects the likelihood. What seems to be a non-identifiability problem is not an issue for sampling since we are interested in inferring the choice probabilities, not the ”true” features. On a more conceptual side, the non-identifiability of the features makes the samples from the posterior hard to interpret. For instance, for the celebrities data one might have hoped to find feature matrices that correspond to a tree or at least find matrices with some other directly interpretable structure. However, although the experiment by Rumelhart and Greeno (1971) was designed with the tree structure in mind Figure 4.16, we do not know the true latent features that lead to the choices of the subjects. Nevertheless, we can use the posterior to predict future choices from past data, assess the similarity of the options and cluster or rank them. The EBA model does not take into account neither the time ordering of choice outcomes nor the identities of the choice makers. For some choice scenarios this information is irrelevant or not available. However, a model that can make use of the additional information when available would potentially be more powerful in representing the structure. This is a direction that the EBA model can be improved. Another possible improvement of the EBA model would be to take into account the shared features as well when calculating the choice probabilities. One possibility is to add to eq. (4.63) the effect of the shared features with a scaling factor s ∈ [0, 1]. The EBA model would be recovered for s = 0, whereas the BTL model would be obtained when the similar features are treated the same as the differences, that is, with s = 1. Using a small but nonzero scale value may improve recovering the latent structure of the alternatives especially when there are pairs of alternatives that have not been compared. 4.6 Discussion The Indian buffet process has been recently introduced by Griffiths and Ghahramani (2005), and rapidly attracted interest due to its conceptual simplicity and promising flexibility for defining nonparametric latent feature models. Its relation to the DP and related distributions has inspired models and inference algorithms from the DP literature to be adjusted to the IBP models. The different approaches with which the equivalent 111
Page 1:
Nonparametric Bayesian Discrete Lat
Page 4 and 5:
Matrizen mit unendlich vielen Spalt
Page 7 and 8:
Contents Zusammenfassung iii Abstra
Page 9:
List of Algorithms 1 Gibbs sampling
Page 13 and 14:
Notation Matrices are capitalized a
Page 15:
Symbol Meaning IBP Z binary latent
Page 18 and 19:
1 Introduction belief in the prior.
Page 20 and 21:
2 Nonparametric Bayesian Analysis b
Page 22 and 23:
2 Nonparametric Bayesian Analysis s
Page 25 and 26:
3 Dirichlet Process Mixture Models
Page 27 and 28:
3.1 The Dirichlet Process the perfo
Page 29 and 30:
α G o G θi x i N 3.1 The Dirichle
Page 31 and 32:
15 10 5 −0.5 0 0.5 2 1 G 0 0 −0
Page 33 and 34:
increment process with the correspo
Page 35 and 36:
α G o π k c i θk x i 8 N 3.1 The
Page 37 and 38:
3.1 The Dirichlet Process Eq. (3.21
Page 39 and 40:
number of components, K number of c
Page 41 and 42:
3.2 MCMC Inference in Dirichlet Pro
Page 43 and 44:
and Bush and MacEachern (1996). 3.2
Page 45 and 46:
3.2.2 Algorithms for non-Conjugate
Page 47 and 48:
Page 49 and 50:
Page 51 and 52:
Page 53 and 54:
Page 55 and 56:
∗ π ∗ π s 3.2 MCMC Inference
Page 57 and 58:
model can be written in the form of
Page 59 and 60:
−1 µ y Σy Σy D ξ normal R 3.3
Page 61 and 62:
the log likelihood term is: where a
Page 63 and 64:
3.3 Empirical Study on the Choice o
Page 65 and 66:
autocovariance coefficient 1 0.8 0.
Page 67 and 68:
# of data points # of data points 5
Page 69 and 70:
3.4 Dirichlet Process Mixtures of F
Page 71 and 72:
µ y Σy ξ R 0 ν w normal µ −1
Page 73 and 74:
ch1 ch2 ch3 ch4 3.4 Dirichlet Proce
Page 75 and 76: 3.4 Dirichlet Process Mixtures of F
Page 77 and 78: # of components # of components # o
Page 79 and 80: ch 2 ch 3 ch 1 ch 2 ch 3 ch 4 3.4 D
Page 81: 3.5 Discussion 3.5 Discussion In th
Page 84 and 85: 4 Indian Buffet Process Models matr
Page 86 and 87: 4 Indian Buffet Process Models In t
Page 88 and 89: 4 Indian Buffet Process Models α z
Page 90 and 91: 4 Indian Buffet Process Models α
Page 92 and 93: 4 Indian Buffet Process Models The
Page 94 and 95: 4 Indian Buffet Process Models colu
Page 96 and 97: 4 Indian Buffet Process Models Pois
Page 98 and 99: 4 Indian Buffet Process Models z α
Page 100 and 101: 4 Indian Buffet Process Models For
Page 102 and 103: 4 Indian Buffet Process Models ciat
Page 104 and 105: 4 Indian Buffet Process Models rati
Page 106 and 107: 4 Indian Buffet Process Models repr
Page 108 and 109: 4 Indian Buffet Process Models samp
Page 110 and 111: 4 Indian Buffet Process Models feat
Page 112 and 113: 4 Indian Buffet Process Models Algo
Page 114 and 115: 4 Indian Buffet Process Models mixi
Page 116 and 117: 4 Indian Buffet Process Models Figu
Page 118 and 119: 4 Indian Buffet Process Models pres
Page 120 and 121: 4 Indian Buffet Process Models ε
Page 122 and 123: 4 Indian Buffet Process Models LL P
Page 124 and 125: 4 Indian Buffet Process Models P+ P
Page 128 and 129: 4 Indian Buffet Process Models dist
Page 130 and 131: 5 Conclusions has been defined as a
Page 132 and 133: A Details of Derivations for the St
Page 135 and 136: B Mathematical Appendix B.1 Dirichl
Page 137 and 138: p3 α 1 0.5 0 0 0.5 0.4 0.3 0.2 0.1
Page 139 and 140: Construction of A Process B.4 Equal
Page 141 and 142: Bibliography D. Aldous. Exchangeabi
Page 143 and 144: Bibliography T. S. Ferguson. Prior
Page 145 and 146: Bibliography L. F. James and J. W.
Page 147 and 148: Bibliography R. M. Neal. Probabilis
Page 149 and 150: Bibliography Y. W. Teh, M. I. Jorda
show all

Nonparametric Bayesian Discrete Latent Variable Models for ...

Create successful ePaper yourself

Delete template?

Save as template?