final report - probability.ca

when the local correlations are sufficiently small so that no phase transitions occur. The Gibbs measure used as
the target distribution for this model is such that the field’s correlations decrease exponentially fast as a function
of distance. In particular, they consider a Gibbs measure on R Zr which has a density with respect to ∏ k∈Z µ (dx k)
r
given by
{
}
exp
− ∑ k∈Z r U k (x)
where U k is a finite-range potential, depending only on a finite number of neighboring terms of k. Markov random
fields with significant phase-transition behavior will require algorithms that use proposals with large jumps in order
to provide sufficient mixing. This will often yield degenerate algorithms, as the acceptance rate will converge to 0,
making the algorithm practically useless.
2.2 Roberts and Rosenthal (2001): & Inhomogeneous Targets
Another validation of the asymptotic optimal acceptance rate equaling 0.234 for the RWM was proven by Roberts
and Rosenthal (2001) [RR01] in the case of inhomogeneous target distributions. Specifically, they considered
inhomogeneous target distributions of the form
d∏
π (x) = C i f (C i x i ) , (2.1)
i=1
where the {C i } are themselves i.i.d. for some fixed distribution, and f is again a fixed one-dimensional density
satisfying the regularity conditions discussed in (1.10). The components in this target are uncorrelated, but heterogeneously
scaled. The target (1.8) used in [RGG97] corresponds to the case where the {C i } are all constant in (2.1).
In order to obtain general results for the limiting theory of the RWM algorithm applied to (2.1), it is assumed that
C i have mean 1 and variance b < ∞.
Theorem 2.1. (Theorem 5 in [RR01]) Consider RWM with target density (2.1), and with proposal distribution
N ( (
0, I d l 2 /d )) , for some l > 0. Consider the time-rescaled process consisting of the first component of the induced
Markov chain, i.e., W (d)
t = C 1 X (1)
[td] . As d → ∞, W t
d ⇒ W t , where W t satisfies the Langevin SDE
dW t = 1 2 g′ (W t ) (C 1 s) 2 dt + (C 1 s) dB t ,
with B t being a standard Brownian motion and where
(
)
s 2 = 2l 2 Φ −lb 1/2 I 1/2 /2
= 1 b · 2 ( l 2 b ) (
Φ − ( l 2 b ) )
1/2
I 1/2 /2 ,
with I = E f
[(g ′ (X)) 2] .
Remark 2.2. When considering a functional of the first coordinate of the algorithm, the efficiency of the algorithm
as a function of acceptance rate is the same as that found in [RGG97], but multiplied with a global “inhomogeneous”
factor C1/b. 2 For a fixed function of f, the optimal asymptotic efficiency is proportional to C1/bd. 2 In particular,
the efficiency of the algorithm is now slowed down by a factor of b = E ( )
Ci
2 /E (Ci ) 2 , which by Jensen’s inequality
is greater than or equal to 1, with equality coming only when the factors C i are all constant, which is equivalent to
the form of (1.8).
Remark 2.3. When the values of C i are known, then instead of using the proposal distribution N ( )
0, σ 2 (d) I d ,
one should use an inhomogeneous proposal which is scaled in proportion to C i for each component, i..e, we should
consider proposals of the form N ( 0, ¯σ 2 (d) diag (C 1 , . . . , C d ) ) . This would then yield a RWM applied to the form
of the target (1.8), since the scaling in the proposal and in the target would cancel each other out. This would
alleviate the inefficiency factor of b as discussed above. See Theorem 6 and the subsequent discussion in [RR01] for
a more detailed assessment of inhomogeneous proposals for such target distributions.
The results above provide some theoretical justification for a frequently used strategy as suggested by Tierney
in [Tie94], which is to use a proposal distribution that is a scalar multiple of the correlation matrix obtained
by estimating the correlation structure of the target distribution, perhaps through an empirical estimate based
on a sample MCMC run, or numerically through curvature calculations on the target distribution itself. Using
inhomogeneous proposals by first obtaining a preliminary estimate of the C i can thus lead to more efficient Metropolis
algorithms.
,
16