Foundations of Data Science

The expectation here is with respect to the “coin tosses” of the algorithm, not with respect
to the underlying distribution p. Let f max denote the maximum absolute value of f. It is
easy to see that
∣∣∣∣∣ ∑
f i p i − E(γ)
∣ ≤ f ∑
max |p i − a ti | = f max ||p − a t || 1 (5.2)
i
i
where the quantity ||p − a t || 1 is the l 1 distance between the probability distributions p
and a t and is often called the “total variation distance” between the distributions. We
will build tools to upper bound ||p − a t || 1 . Since p is the stationary distribution, the t for
which ||p − a t || 1 becomes small is determined by the rate of convergence of the Markov
chain to its steady state.
The following proposition is often useful.
Proposition 5.4 For two probability distributions p and q,
||p − q|| 1 = 2 ∑ i
(p i − q i ) + = 2 ∑ i
(q i − p i ) +
where x + = x if x ≥ 0 and x + = 0 if x < 0.
The proof is left as an exercise.
5.2.1 Metropolis-Hasting Algorithm
The Metropolis-Hasting algorithm is a general method to design a Markov chain whose
stationary distribution is a given target distribution p. Start with a connected undirected
graph G on the set of states. If the states are the lattice points (x 1 , x 2 , . . . , x d ) in R d
with x i ∈ {0, 1, 2, , . . . , n}, then G could be the lattice graph with 2d coordinate edges at
each interior vertex. In general, let r be the maximum degree of any vertex of G. The
transitions of the Markov chain are defined as follows. At state i select neighbor j with
probability 1 . Since the degree of i may be less than r, with some probability no edge
r
is selected and the walk remains at i. If a neighbor j is selected and p j ≥ p i , go to j. If
p j < p i , go to j with probability p j /p i and stay at i with probability 1 − p j
p i
. Intuitively,
this favors “heavier” states with higher p values. So, for i, adjacent to j in G,
p ij = 1 (
r min 1, p )
j
p i
and
Thus,
p ii = 1 − ∑ j≠i
p i p ij = p i
r min (
1, p j
p i
)
= 1 r min(p i, p j ) = p j
r min (
1, p i
p j
)
= p j p ji .
By Lemma 5.3, the stationary probabilities are indeed p i as desired.
146
p ij .