Theoretical Neuroscience

Model GenerativeModel RecognitionModel LearningRules
mixtureof
Gaussians
factor
analysis
principal
components
analysis
sparse
coding
independent
components
analysis
binary
Helmholtz
machine
P[v;G] = γ v
P[u|v;G] = N (u;g v , v )
P[v;G] = N (v;0,1)
P[u|v;G] = N (u;G·v,)
=diag ( 1 , 2 ,..., Nu
)
P[v;G] = N (v;0,1)
u =G·v
P[v;G] ∝ ∏ a exp(g(v a))
P[u|v;G] = N (u;G·v,)
P[v;G] ∝ ∏ a exp(g(v a))
u =G·v
P[v;G] = ∏ (
a f(ga ) ) ( va
1 −f(g a ) ) 1−va
P[u|v;G] = ∏ b(
fb (h +G·v) ) ub
×
(
1 −fb (h +G·v) ) 1−ub
f b (h +G·v) =f ( h b +[G·v] b
)
P[v|u;G] ∝ γ v N (u;g v , v )
P[v|u;G] = N (v;W·u,) G
(
= I +G T·−1·G
−1
W = ·G T·
v =W·u G
T
W = (G T·G) −1·G
G T·(u −G·v) +g ′ (v) =0
) −1
µ v =〈P[v|u;G]〉
g v =〈P[v|u;G]u〉/γ v
v =〈P[v|u;G]|u −g v | 2 〉/(N u γ v )
=C·W T·(W·C·W T + ) −1
=diag
(G··G T + (I −G·W)·C·(I −G·W) T)
C =〈uu〉
=C·W T·(W·C·W T ) −1
C =〈uu〉
G →G + ǫ(u −G·v)v
) (∑ )( → 〈v
2
a −〈v a 〉 2 〉/σ 2) 0.01
(∑
b G2 ba
b G2 ba
v =W·u W ab →W ab + ǫ ( W ab +g ′ )
(v a )[v·W] b
W =G −1 g ′ (v) =−tanh(v)ifg(v) =−lncosh(v)
Q[v;u,W] = ∏ a(
fa (w +W·u) ) va
×
(
1 −fa (w +W·u) ) 1−va
f a (w +W·u) =f ( w a +[W·u] a
)
wake:u ∼P[u],v ∼Q[v;u,W]
g →g + ǫ(v −f(g))
h →h + ǫ(u −f(h +G·v))
G →G + ǫ(u −f(h +G·v))v
sleep:v ∼P[v;G],u ∼P[u|v;G]
w →w + ǫ(v −f(w +W·u))
W →W+ ǫ(v −f(w +W·u))u
Table1:Allmodelsarediscussedindetailinthetext,andtheformsquotedarejustforthesimplestcases. N (u;g,)isamultivariateGaussian
distributionwithmeangandcovariancematrix (for N (u;g,),thevarianceofeachcomponentis ).Forthesparsecodingnetwork, σ 2 isa
targetforthevariancesofeachoutputunit.FortheHelmholtzmachine,f(c) =1/(1 +exp(−c)),andthesymbol ∼indicatesthattheindicated
variableisdrawnfromtheindicateddistribution.Othersymbolsanddistributionsaredefinedinthetext.
Συµµαρψ οφ Χαυσαλ Μοδελσ
10.6 Αππενδιξ
10.6 Αππενδιξ 395