Foundations of Data Science

5.3 Areas and Volumes
Computing areas and volumes is a classical problem. For many regular figures in
two and three dimensions there are closed form formulae. In Chapter 2, we saw how to
compute volume of a high dimensional sphere by integration. For general convex sets in
d-space, there are no closed form formulae. Can we estimate volumes of d-dimensional
convex sets in time that grows as a polynomial function of d? The MCMC method answes
this question in the affirmative.
One way to estimate the area of the region is to enclose it in a rectangle and estimate
the ratio of the area of the region to the area of the rectangle by picking random points
in the rectangle and seeing what proportion land in the region. Such methods fail in high
dimensions. Even for a sphere in high dimension, a cube enclosing the sphere has exponentially
larger area, so exponentially many samples are required to estimate the volume
of the sphere.
It turns out that the problem of estimating volumes of sets is reducible to the problem
of drawing uniform random samples from sets. Suppose one wants to estimate the volume
of a convex set R. Create a concentric series of larger and larger spheres S 1 , S 2 , . . . , S k
such that S 1 is contained in R and S k contains R. Then
Vol(R) = Vol(S k ∩ R) = Vol(S k ∩ R) Vol(S k−1 ∩ R)
Vol(S k−1 ∩ R) Vol(S k−2 ∩ R) · · · Vol(S 2 ∩ R)
Vol(S 1 ∩ R) Vol(S 1)
If the radius of the sphere S i is 1 + 1 d times the radius of the sphere S i−1, then the value
of
Vol(S k−1 ∩ R)
Vol(S k−2 ∩ R)
can be estimated by rejection sampling provided one can select points at random from a
d-dimensional region. Since the radii of the spheres grows as 1+ 1 , the volume of a sphere
d
is ( 1 + d) 1 d
< e times the volume of the preceding sphere and the number of spheres is at
most
O(log 1+(1/d) r) = O(rd)
where r is the ratio of the radius of S k to the radius of S 1 .
It remains to show how to draw a uniform random sample from a d-dimensional set.
It is at this point that we require the set to be convex so that the Markov chain technique
will converge quickly to its stationary probability. To select a random sample from a
d-dimensional convex set, impose a grid on the region and do a random walk on the grid
points. At each time, pick one of the 2d coordinate neighbors of the current grid point,
each with probability 1/(2d) and go to the neighbor if it is still in the set; otherwise, stay
put and repeat. If the grid length in each of the d coordinate directions is at most some
a, the total number of grid points in the set is at most a d . Although this is exponential in
d, the Markov chain turns out to be rapidly mixing (the proof is beyond our scope here)
150