Foundations of Data Science

in time polynomial in n.
A quantity called the mixing time, loosely defined as the time needed to get close to
the stationary distribution, is often much smaller than the number of states. In Section
5.4, we relate the mixing time to a combinatorial notion called normalized conductance
and derive upper bounds on the mixing time in several cases.
5.1 Stationary Distribution
Let p t be the probability distribution after t steps of a random walk.
long-term average probability distribution a t by
Define the
a t = 1 t (p 0 + p 1 + · · · + p t−1 ) .
The fundamental theorem of Markov chains asserts that for a connected Markov chain, a t
converges to a limit probability vector x, which satisfies the equations xP = x; ∑ i x i = 1
which we can rewrite as
x[P − I, 1] = [0, 1].
We will prove now that the matrix [P −I, 1] has rank n provided the Markov Chain is connected.
This implies that there is a unique solution to the equations x[P − I, 1] = [0, 1].
We denote this solution by π. It has non-negative components and so is a probability
vector. Since πP = π, we have that running one step of the Markov Chain starting with
distribution π leaves us in the same distribution. Thus also running any number of steps
of the Markov Chain starting the first step with π leaves the distribution still the same.
For this reason, π is called the stationary distribution.
Lemma 5.1 Let P be the transition probability matrix for a connected Markov chain.
The n × (n + 1) matrix A = [P − I , 1] obtained by augmenting the matrix P − I with an
additional column of ones has rank n.
Proof: If the rank of A = [P − I, 1] was less than n there would be two linearly independent
solutions to Ax = 0. Each row in P sums to one so each row in P − I sums to
zero. Thus (1, 0), where all but the last coordinate is 1, is one solution. Assume there
was a second solution (x, α) perpendicular to (1, 0). Then (P − I)x + α1 = 0. Thus for
each i, ∑ p ij x j − x i + α = 0 or x i = ∑ j p ijx j + α. Each x i is a convex combination
j
of some of the x j plus α. Since x is perpendicular to 1, not all x i can be equal. Let
S = {i : x i = Max n j=1x j } be the set of i for which x i attains its maximum value. ¯S is
not empty. Connectedness implies that there is some edge (k, l), k ∈ S, l ∈ ¯S. Thus,
x k > ∑ j p kjx j . Therefore α must be greater than 0 in x k = ∑ j p kjx j + α. A symmetric
argument with the set of i with x i taking its minimum value implies α < 0 producing a
contradiction proving the lemma.
143