Foundations of Data Science

to k ≥ 4, but with different constants. It turns out that the second moment method
cannot be directly applied to get a lower bound on r 3 because the variance is too high. A
simple algorithm, called the Smallest Clause Heuristic (abbreviated SC), yields a satisfying
assignment with probability tending to one if c < 2, proving that r 3 3 ≥ 2 . Other more
3
difficult to analyze algorithms, push the lower bound on r 3 higher.
The Smallest Clause Heuristic repeatedly executes the following. Assign true to a
random literal in a random smallest length clause and delete the clause since it is now
satisfied. In more detail, pick at random a 1-literal clause, if one exists, and set that
literal to true. If there is no 1-literal clause, pick a 2-literal clause, select one of its two
literals and set the literal to true. Otherwise, pick a 3-literal clause and a literal in it and
set the literal to true. If we encounter a 0-length clause, then we have failed to find a
satisfying assignment; otherwise, we have found one.
A related heuristic, called the Unit Clause Heuristic, selects a random clause with one
literal, if there is one, and sets the literal in it to true. Otherwise, it picks a random as
yet unset literal and sets it to true. Another variation is the “pure literal” heuristic. It
sets a random “pure literal”, a literal whose negation does not occur in any clause, to
true, if there are any pure literals; otherwise, it sets a random literal to true.
When a literal w is set to true, all clauses containing w are deleted, since they are
satisfied, and ¯w is deleted from any clause containing ¯w. If a clause is reduced to length
zero (no literals), then the algorithm has failed to find a satisfying assignment to the
formula. The formula may, in fact, be satisfiable, but the algorithm has failed.
Example: Consider a 3-CNF formula with n variables and cn clauses. With n variables
there are 2n literals, since a variable and its complement are distinct literals. The expected
number of times a literal occurs is calculated as follows. Each clause has three literals.
Thus, each of the 2n different literals occurs (3cn) = 3 c times on average. Suppose c = 5.
2n 2
Then each literal appears 7.5 times on average. If one sets a literal to true, one would
expect to satisfy 7.5 clauses. However, this process is not repeatable since after setting a
literal to true there is conditioning so that the formula is no longer random.
Theorem 4.20 If the number of clauses in a random 3-CNF formula grows as cn where
c is a constant less than 2/3, then with probability 1 − o(1), the Shortest Clause (SC)
Heuristic finds a satisfying assignment.
The proof of this theorem will take the rest of the section. A general impediment to
proving that simple algorithms work for random instances of many problems is conditioning.
At the start, the input is random and has properties enjoyed by random instances.
But, as the algorithm is executed, the data is no longer random; it is conditioned on the
steps of the algorithm so far. In the case of SC and other heuristics for finding a satisfying
assignment for a Boolean formula, the argument to deal with conditioning is relatively
110