Notes on computational linguistics.pdf - UCLA Department of ...

Stabler - Lx 185/209 2003
8.1.13 Controversies
(98) Consider the following quote from Charniak (1993, p32):
After adding the probabilities, we could call this a “probabilistic finite-state automaton,” but
such models have different names in the statistical literature. In particular, that in figure 2.4 is
called a Markov chain.
.5 the
.5 here
.5 dog
.5 a .5 cat
.5 there
Figure 2.4 A trivial model of a fragment of English
.5 ate
.5 slept
Like finite-state automata, Markov chains can be thought of as acceptors or generators. However,
associated with each arc is the probability of taking that arc given that one is in the state
at the tail of the arc. Thus the numbers associated with all of the arcs leaving a state must sum
to one. The probability then of generating a given string in such a model is just the product
of the probabilities of the arcs traversed in generating the string. Equivalently, as an acceptor
the Markov chain assigns a probability to the string it is given. (This only works if all states are
accepting states, something standardly assumed for Markov processes.)
This paragraph is misleading with respect to one of the fundamental points in this set of notes:
The figure shows a finite automaton, not a Markov chain or Markov model.
While Markov models are similar to probabilistic finite automata in the sense mentioned in (78), Markov
chains are not like finite automata, as we saw in (77).
In particular, Markov models can define output sequences with (a finite number of) unbounded dependencies,
but Markov chains define only state sequences with the Markovian requirement that blocks
non-adjacent dependencies.
(99) Charniak (1993, p39) says:
One of the least sophisticated but most durable of the statistical models of English is the ngram
model. This model makes the drastic assumption that only the previous n-1 words have
any effect on the probabilities for the next word. While this is clearly false, as a simplifying
assumption it often does a serviceable job.
Serviceable? What is the job??
For the development of the science and of the field, the question is: how can we move towards a model
that is not “clearly false.”
149