Measuring the Goals and Incentives of Local Chinese Officials

this character against the lexicon to see if C 1 is a uni-gram word. Then, it checks to see
if C 1 C 2 is a bi-gram word, and continues to do so until the sequence of characters is
longer than the longest sequence containing that string of characters in the lexicon. The
most plausible word is the longest, or maximum, match. If this match were C 1 C 2 , the
algorithm would repeat the process starting with C 3 (Chen and Liu, 1992; Cheng, Young
and Wong, 1999).
Complex maximum matching finds all possible combinations of three-word sequence
starting with C 1 and takes the most plausible word as the one belonging to the three-word
sequence containing the most characters in total. For example, in the string given above,
let’s say there are three possible three-word chunks:
C 1 C 2 C 3 C 4
C 1 C 2 C 3 C 4 C 5
C 1 C 2 C 3 C 4 C 5 C 6
The three-word sequence with the longest length is third combination, and as a result,
C 1 C 2 will be considered the correct word. The algorithm will then move to C 3 and
restart this process.
Based on the segmentation, a dictionary of 77,855 unique words emerged, and a total
of 1,612 stopwords were identified. Stopwords included parts of speech that do not convey
substantive content, for example particles, prepositions, pronouns, conjunctions, and noun
classifiers. This list is based on stopwords used by baidu.com, China’s largest internet
search engine. 15 Noun classifiers show the conceptual classification of the referent of
the noun. For example, in Chinese, the noun classifier for humans is “ge” such that “3
teachers” is “3-ge teachers,” the noun classifer for birds is “zhi” so that “3 birds” is “3-zhi
birds,” and the noun classifier for things that are large and thin such pieces of paper or a
table top is “zhang” such that “3 tables” is “3-zhang tables.” There are hundreds of noun
classifiers in Chinese. Words that appeared in more than 95% (49,641) of documents were
also removed. Perhaps removing these extremes is redundant with removing stopwords,
which are words that appear at high frequency, but I veered on the side of conservatism to
ensure that words appearing frequently on websites, such as “web,” “web page,” “back,”
15 Downloaded from http://wenku.baidu.com/view/982a25c608a1284ac85043fa.html.
25