Introduction to Computational Linguistics

14. Digression: Time Complexity 52
would lead one to believe. Here is another algorithm: call a state q n–reachable if
⃗x
there is a word of length ≤ n such that i 0 → q; call it reachable if it is n–reachable
for some n ∈ ω. L(A) is nonempty if some q ∈ F is reachable. Now, denote by
R n the set of n–reachable states. R 0 := {i 0 }. In the nth step we compute R n+1 by
taking all successors of points in R n . If R n+1 = R n we are done. (Checking this
takes |Q| steps. Alternatively, we need to iterate the construction at most |Q| − 1
times. Because if at each step we add some state, then R n grows by at least 1, so
this can happen only |Q| − 1 times, for then R n has at least |Q| many elements.
On the other hand, it is a subset of Q, so it can never have more elements.) Let
us now see how much time we need. Each step takes R n × |A| ≤ |Q| × |A| steps.
Since n ≤ |Q|, we need at most |Q| − 1 steps. So, we need c · |A||Q| 2 steps for some
constant c.
Consider however an algorithm that takes n 2 steps.
(135)
n n 2 n n 2
1 1 11 121
2 4 12 144
3 9 13 169
4 16 14 196
5 25 15 225
6 36 16 256
7 49 17 289
8 64 18 324
9 81 19 361
10 100 20 400
For n = 10 it takes 100 steps, or a tenth of a millisecond, for n = 20 it takes 400,
and for n = 30 only 900, roughly a millisecond. Only if you double the size of the
input, the number of steps quadruple. Or, if you want the number of steps to grow
by a factor 1024, you have to multiply the length of the input by 32! An input of
even a thousand creates a need of only one million steps and takes a second on
our machine.
The algorithm described above is not optimal. There is an algorithm that is
even faster: It goes as follows. We start with R −1 := ∅ and S 0 = {q 0 }. Next, S 1
is the set of successors of S 0 minus R 0 , and R 0 := S 0 . In each step, we have two
sets, S i and R i−1 . R i−1 is the set of i−1–reachable points, and S i is the set of points
that can be reached from it it one step, but which are not in R i−1 . In the next step