Monte Carlo Inference - STAT - EPFL

Dimension-matching moves□ We want to compute the probability of jumping from point u (1) ∈ R n 1to u (2) ∈ R n 2, wheren 1 ≠ n 2 . To do so, we introduce– auxiliary variables w 1 ∈ R m 1, w 2 ∈ R m 2, whose dimensions are chosen so thatn 1 + m 1 = n 2 + m 2– a random variable t which gives a bijection between (u (1) ,w 1 ) ↔ (u (2) ,w 2 ),t = t 1 (u (1) ,w 1 ) = t 2 (u (2) ,w 2 ) ∈ R n 1+m 1□ Then the probability π(u)q(v | u) is replaced byπ{(1,u (1) ) | y} × q(1,u (1) ∂(u (1) ,w 1 )) × p 1 (w 1 ) ×∣ ∂t ∣where the terms are– the posterior probability of being at u (1) in R n 1– the probability of proposing a move away from this point– the density of w 1– the Jacobian transforming the preceding terms into a density for tMonte Carlo Inference Spring 2009 – slide 202Acceptance probability□ The probability π(v)q(u | v) is likewise replaced byπ{(2,u (2) ) | y} × q(2,u (2) ∂(u (2) ,w 2 )) × p 2 (w 2 ) ×∣ ∂t ∣so the acceptance probability ratio π(v)q(u | v)/{π(u)q(v | u)} for the proposed move from(u (1) ,w 1 ) ↦→ (u (2) ,w 2 ) isπ{(2,u (2) ) | y}q(2,u (2) )p 2 (w 2 )∂(u (2) ,w 2 )π{(1,u (1) ) | y}q(1,u (1) )p 1 (w 1 ) ∣∂(u (1) ,w 1 ) ∣□ Often in practice the moves are set up so that m 1 = 0 or m 2 = 0, in which case there is no needto generate w 1 or w 2 . For example, if m 2 = 0, then the acceptance probability for theMetropolis–Hastings step ismin{1,where w ≡ w 1 .π{(2,u (2) ) | y}q(2,u (2) )π{(1,u (1) ) | y}q(1,u (1) )p 1 (w)}∂(u (2) )∣∂(u (1) , (10),w) ∣Monte Carlo Inference Spring 2009 – slide 203200