Algorithm Design

684
Chapter 12 Local Search
utting the two edges incident
t~obad
s would
local
be
optimum:
better.
-
Figure 12.5 An instance of the Minimum s-t Cut Problem, where all edges have
capacity 1.
exponentiallY many neighbors. This appears to be contrary to the general rule
that "the neighborhood of a solution should not be too large;’ as stated in
Section 12.5. However, we will be working with neighborhoods in a more
subtle way here. Keeping the size of the neighborhood small is good if the
plan is to search for an improving local step by brute force; here, however, we
will use a polynomial-time minimum-cut computation to determine whether
any of a solution’s exponentially many neighbors represent an improvement.
The idea of the local search is to use our polynomial-time algorithm
for Two-Label Image Segmentation to find improving local steps. First let’s
consider a basic implementation of this idea that does not always give a good
approximation guarantee. For a labeling f, we pick two labels a, b ~ L and
restrict attention to the nodes that have labels a or b in labeling f. In a single
local step, we will allow any subset of these nodes to flip labels from a to b, or
from b to a. More formally, two labelings f and f’ are neighbors if there are two
labels a, b ~ L such that for all other labels c {a, b} and all nodes i ~ V, we
have f(i) = c if and only if f’(i) = c. Note that a state f can have exponentially
many neighbors, as an arbitrary subset of the nodes labeled a and b can flip
their label. However, we have the following.
(12.8)
If a labeling f is not locally optimal for the neighborhood above, then a
neighbor with smaller penalty can be found via k2 minimum-cat computations.
ProoL There are fewer than k2 pairs of distinct labels, so we can try each pair
separately. Given a pair of labels a, b ~ L, consider the problem of finding an
improved labeling via swapping labels of nodes between labels a and b. This
is exactly the Segmentation Problem for two labels on the subgraph of nodes
that f labels a or b. We use the algorithm developed for Twd-Label Image
Segmentation to find the best such relabefing. "
12.6 Classification via Local Search
Figure 12.6 A bad local optimum for the local search algorithm that considers only
two labels at a time.
This neighborhood is much better than the single-flip neighborhood we
considered first. For example, it solves the case of two labels optimally.
However, even with this improved neighborhood, local optima can still be
bad, as shown in Figure 12.6. In this example, there are three nodes s, t, and z
that are each required to keep their initial labels. Each other node lies on one of
the sides of the triangle; it has to get one of the two labels associated with the
nodes at the ends of this side. These requirements can be expressed simply by
giving each node a very large assignment penalty for the labels that we are not
allowing. We define the edge separation penalties as follows: The light edges
in th e figure have penalty 1, while the heavy edges have a large separation
penalty of M. Now observe that the labeling in the figure has penalty M + 3
but is locally optimal. The (globally) optimal penalty is only 3 and is obtained
from the labeling in the figure by relabeling both nodes next to s.
A Local Search Neighborhood That Works Next we define a different neighborhood
that leads to a good approximation algorithm. The local optimum in
Figure 12.6 may be suggestive of what would be a good neighborhood: We
need to be able to relabel nodes of different labels in a single step. The key is
to find a neighbor relation rich enough to have this property, yet one that still
allows us to find an improving local step in polynomial time.
Consider a labeling f. As part of a local step in our new algorithm, we will
want to do the following. We pick one labe! a ~ L and restrict attention to the
685