Pushdown Automaton: CFL and the pumping lemma

Unit 9
More Pushdown Automata
Context-free Languages
Pumping Lemma for CFL
Reading: Sipser, chapter 2.3
1

Properties of PDAs
• An NFA can only distinguish between |Q|
different characterizations.
• A PDA can distinguish between an unlimited
number of characterizations.
• A PDA can recognize non-regular languages
because the stack can ‘count’.
• A PDA can count ‘more than once’. But it can
not mix counters. Only one active counter can
be used each time.
2

Example: L = {a i b i c j d j |i,j0}
Construct a PDA to recognize:
L = {a i b i c j d j |i,j0}
We will use the empty stack model.
The basic idea:
• Push to the stack an A for each a, pop an A
for each b.
• Push to the stack a C for each c, pop a C for
each d .
3

Properties of PDAs
• We had two ways to describe regular
languages:
Regular-Expressions
syntactic
DFA / NFA
computational
• How about context-free-languages
CFG
syntactic
PDA
computational
4

CFG=PDA
Theorem: A language is context-free iff
some pushdown automaton recognizes it.
Proof:
• CFLPDA: we show that if L is CFL
then a PDA recognizes it.
• PDA CFL: we show that if a PDA
recognizes L then L is CFL.
5

From CFG to PDA
• Proof idea: Use PDA to simulate leftmost
derivations.
• Leftmost derivation : A derivation of a string is
a leftmost derivation if at every step the leftmost
remaining variable is the one replaced.
• We use the stack to store the suffix that has not
been derived so far.
• Any terminal symbols appearing before the
leftmost variable are matched right away.
6

Different derivations for the same
CFG: EEE | E+E
E0 | 1 | 2 | … | 9
E
parse tree
leftmost
derivation
E
E
E
E E E
5 E E
5 3
E
5 3
2
E
E
E
E
rightmost
derivation
E
E
E
E
2
E E 2
E 3
2
5 3
2
5 + 3 x 2
7

Starting configuration:
control
input:
5 + 3 x 2 E
E
stack:
$
control
E EE
input:
stack:
5 + 3 x 2 E
EE
x
E
$

control
E
E
E
E E E
input:
stack:
5 + 3 x 2 E
E+EE
+
E
x
E
$
control
E
EE
E E E
5
E E
input:
5 + 3 x 2
5+EE
stack:
5
+
E
x
E
$

control
E
EE
E E E
5
E E
input:
stack:
5 + 3 x 2 E
EE
x
E
$
E
E
E
control
E E E
5 E E
5 3
E
input:
stack:
5 + 3 x 2 3
3E
x
E
$

E
E
E
E E E
control
5 E E
5 3
E
input:
stack:
5 + 3 x 2 E
$
E
E
E
E
E E E
control
5 E E
5 3
E
5 3
2
input:
5 + 3 x 2
2
stack:
2
$

E
E
E
E E E
control
5 E E
5 3
E
5 3
2
input:
stack:
5 + 3 x 2 $
The string ‘5 + 3 x 2’ is accepted

From CFG to PDA
Informally:
1. Place the marker symbol $ and the start variable
S on the stack.
2. Repeat the following steps:
– If the top of the stack is a variable A:
Choose a rule A→ 1 … k and substitute A with 1 … k
– If the top of the stack is a terminal a:
Read next input symbol and compare to a
If they don’t match, reject (die)
– If top of stack is $, go to accept state
13

From CFG to PDA
• For a given CFG G=(V,,S,R),
we construct a PDA P=(Q,,,,q 0 ,F) where:
– Q={q start , q loop , q accpt }
– = V{$}
– q 0 =q start
– F={q accpt }
14

From CFG to PDA
• We define as follows (shorthand notation):
– (q start ,,)={(q loop ,S$)}
– (q loop ,,A)={(q loop , 1 … k ) | for each A 1 … k in R}
– (q loop ,a,a)={(q loop ,) | for each a }
– (q loop ,,$)={(q accpt, )}
{,A 1 … k | for rules A 1 … k }
{a,a | for all a}
,S$
,$
q start q loop q accpt
15

Example:
• Construct a PDA for the following CFG G:
SaTb | b
TTa |
L(G)= a * b
,SaTb
,Sb
,TTa
,T
a,a
b,b
,S$
,$
q start q loop q accpt
16

From PDA to CFG
context-free grammar
✓
pushdown automaton
• First, we simplify the PDA:
– It has a single accept state q f
– $ is always popped exactly before accepting
– Each transition is either a push, or a pop, but
not both
17

• single accept state q f :
From PDA to CFG
,
,
18

From PDA to CFG
• $ is always popped exactly before accepting:
{,A | A, A$}
,$
19

From PDA to CFG
• Each transition is either a push, or a pop:
,ab ,a ,b
z
, ,z ,z
20

From PDA to CFG
• For any word w accepted by a PDA
P=(Q,,,,q 0 ,q f ) the process starts at q 0 with an
empty stack and ends at q f with an empty stack.
• Definition: for any two states p,qQ we define
L p,q to be the language that if we starts at p with
an empty stack and run on wL p,q we end at q
with an empty stack.
• We define for L p,q a variables A p,q s.t.
L p,q = {w | A p,q * w}
• Note, that L(P)=L q0 ,q f
21

From PDA to CFG
• Consider a word wL p,q
• While running w on P, the stack is empty at p and
at q but what happens in the middle
• Two possibilities:
– Option 1: The stack also empty in the middle
– Option 2: The stack never empty in the middle
stack
height
p
r q p q
22

From PDA to CFG
Option 1: The stack also empty in the middle
• If the stack become empty at some state r then the
word wL pq can be reconstructed by a
concatenation of a word from L pr and a word from
L rq , thus L pr L rq L pq
• In the CFG we express this by a rule: A pq A pr A rq
p
r
q
generated by A pr
generated by A rq
23

From PDA to CFG
Option 2: The stack never empty in the middle
• The symbol that has been pushed at p is the
symbol that is popped at q.
• Thus, if at p we read a symbol a and moved to r,
while from state s we read a symbol b and moved
to q, aL r,s bL p,q and in CFG we have A pq aA rs b
r
p
a
s q
b
24
generated by A rs

From PDA to CFG
Let P=(Q, , , , q 0 , q f ) a given PDA.
We construct a CFL G=(V,,S,R) as follows*:
• V = {A p,q | p,qQ}
• S=A
q 0 ,q f
• R is a set of rules constructed as follows:
* Proof of correctness and further reading at the supplementary
material in the course web page .
25

From PDA to CFG
• Add the following rules to R:
1. For each p,q,r,sQ, t, and a,b ,
if (r,t)(p,a,) and (q,)(s,b,t) add a rule
A pq aA r,s b
p
a,t
r
s
b,t
q
push t
pop t
2. For each p,q,rQ, add a rule A p,q A p,r A r,q
p r q
1. For each pQ, add the rule A p,p
p
26

Example:
0,A
,$ #,
q s q 0
1,A
q 1
,$
q 2
0,A
1,A
,$ #,z
q s q 0
q 3
,z
q 1
,$
q 2
L(P)=0 n #1 n
27

Example:
0,A
1,A
,$ #,z
q s q 0
q 3
,z
q 1
,$
q 2
start variable: A S2
productions:
A SS → A SS A SS
A SS → A S0 A 0S
A SS → A S1 A 1S
A SS → A S2 A 2S
A S1 → A SS A S1
A SS →
A 00 →
A 11 →
A 22 →
A 33 →
A S2 → A 01
A 01 → 0 A 01 1
A 01 → #A 33
A S1 → A S0 A 01
A S1 → A S1 A 11
...
28

CFG=PDA
• We have shown that a language is context-free
iff some pushdown automaton recognizes it.
• In particular all regular languages can be
generated by CFGs and so can be recognized
by PDA.
• The class of languages accepted by nondeterministic
PDAs is larger than those
accepted by deterministic PDAs.
29

The Context-free Languages
DPDA
the regular languages
context-free languages
30

Non context-free Languages
• Consider the language L={a i b i c i |i0}.
• When trying to build a push-down automaton
that recognizes L, we can compare the number
of a-'s with b-'s or c-'s but not both;
• If we compared the number of a-'s to the
number of b-'s then we can't compare c-'s with
any of them, as at this stage the stack (or
counter) is empty.
31

Non context-free Languages
• So some languages seem to be not CFL.
• The question is which
• This can be determined using the pumping
lemma for context-free languages.
32

The Pumping Lemma - background
• Let L be a CFL and let G be a simple
grammar (no unit/ rules) generating it.
• Let wL be a long enough word (we will say
later what is long).
• The parsing tree of w contains a long path
from S to some leaf (terminal).
• On this long path some variable R must
repeat (remember, w is long).
33

The Pumping Lemma - background
S
R
• Divide w into uvxyz
according to the parse
tree, as in the figure.
• Each occurrence of R
has a subtree under it.
R
u v x y z
34

The Pumping Lemma - background
• The upper occurrence of R has a larger subtree
and generates vxy.
• The lower occurrence of R has a smaller
subtree and generates only x.
• Both subtrees are generated by the same
variable R.
• That means if we substitute one for the other we
will still obtain valid parse trees.
S
R
R
u v x y z
35

S
S
R
R
u v y z
u
R
x
z
v
x
y
Replacing the smaller by
the larger repeatedly
generates the string
uv i xy i z at each i>0.
Replacing the larger by
the smaller generates the
string uxz or uv i xy i z
where i=0.
Therefore, for all i0, w i = uv i xy i z is also in L
36

The Pumping Length
• That means that every CFL has a special
value called the pumping length such that all
strings longer than the pumping length can
be "pumped".
• The string can be divided into 5 parts
w=uvxyz.
• The second and fourth can be pumped to
produce additional words in L.
• for all k0, w k = uv k xy k z can also be
generated by the grammar.
37

Pumping Lemma for CFL
Lemma: Let L be a context-free language.
There is a positive integer p (the pumping
length) such that for all strings wL with
|w|p, w can be divided into five pieces
w=uvxyz satisfying the following conditions:
1. |vy|>0
2. |vxy|p
3. for each i0, uv i xy i zL
38

Proof - value of p
• First we find out the value of p.
• Let G be a CFG for CFL L.
• Let b be the maximum number of symbols in
the right side of any rule in G.
• So we know that in any parse tree of G a node
can't have more than b children.
• So if the height of a parsing tree for wL is h
then |w|< b h
(h>log b |w|).
A
A 1 2 3 b
1 2 3 b
39

Proof - value of p
• Let |V| be the number of variables in G.
• We set p = b |V|+2 .
(h>log b |p|).
• Then for any string of length p the parse tree
requires height at least |V|+2 (Note, b>1 since
there are no unit rules).
• Given a string wL, s.t. |w| p , since G has
only |V| variables, at least one of the variables
repeats (height |V|+2 |V|+1 variables +
terminal).
• W.l.o.g. assume this variable is R
40

Proof – condition 1
• To prove condition 1 (|vy|>0) we have to show it
is impossible that both v and y are .
• We use a grammar without unit rules.
• But the only way to have v=y=, is to have a
rule R R, which is a unit rule. Contradiction.
• So condition 1 is satisfied.
S
R
R
u v x y z
41

Proof – condition 2
• To prove condition 2 (|vxy|p) we will check
the height of the subtree rooted in first R =
the subtree that generates vxy.
• Its height is at most |V|+2 (R was selected as
a variable that has two occurrences within
the bottom |V|+1 levels of the parsing tree).
• So it can generate a string of length at most
b |V|+2 .
• Since p= b |V|+2 , condition 2 is satisfied.
42

Proof – condition 3
S
S
R
R
R
x
u v R y z
u
z
v
Replacing the smaller by
the larger repeatedly
generates the string
uv i xy i z at each i>0.
x
y
Replacing the larger by
the smaller generates
the string uxz or uv i xy i z
where i=0.
Therefore, for all i0, w i =uv i xy i z is also in L
43

Usage of the lemma
• We use the pumping lemma to prove that a
language is not context-free.
General Structure:
• Assume (by contradiction) that L is contextfree
and therefore should fulfill the lemma.
• Let p be the pumping length for L.
• Select a word wL s.t. |w|>p.
• Show that for any partition of w into five parts
uvxyz such that |vy|>0 and |vxy|p, there
exists an i such that w i = uv i xy i z L
• Contradiction!
44

Usage of the lemma - Example
• We use the pumping lemma to prove that the
language L={a n b n c n |n0} is not context-free.
Proof:
• Assume, by contradiction, that L is context-free
thus satisfying the lemma.
• Let p be the pumping length for L.
• We select the string w= a p b p c p . wL and |w|>p,
so it can be pumped.
45

L={a n b n c n |n0}
• Divide w into five parts uvxyz such that |vy|>0
and |vxy|p.
• There are two cases:
1. u and y are both homogeneous.
2. v or y is heterogeneous.
46

L
n n n
p p p
{ a b c | n 0}. Let w a b c uvxyz.
a p b p c p
Case 1:
v
y
v and y are
homogeneous
v
y
v
y
v
y
v
y
Case 2
v or y is
heterogeneous
or
v
y
47

L={a n b n c n |n0}
Case 1: v and y contain only one type of
alphabet symbol. By choosing i=2 we get
w 2 =uv 2 xy 2 z in which the number of
appearances of one or two symbols
increased while the third symbol remain
unchanged. So w 2 cannot contain the same
number of a's, b's and c's and w 2 L.
48

L={a i b i c i |i0}
Case 2: v or y contain two types of alphabet
symbols. (Cannot have three because
condition 2 holds). So if we choose i=2 then
the order of the symbols in v 2 or y 2 was
destroyed and w 2 =uv 2 xy 2 z L.
Conclusion: The assumption that L is contextfree
is false. and L={a n b n c n |n0} is not CFL.
49

Pumping Lemma – another example
Let L={ww| w{0,1}*}
[ 00, 110110L ; 010, 0010L ]
Prove that L is not context-free
Proof: In class.
50

Intersection 1
CFL are not closed under intersection.
Proof: by contradiction
L 1 ={a n b n c k |n,k>0}
L 2 ={a k b n c n | n,k>0}
Both L 1 and L 2 are CFL but
L 1 L 2 = {a j b j c j | j>0} is not a CFL.
51

Intersection 2
• CFL and RL are closed under intersection.
Proof:
Build a product automaton of a PDA and a DFA:
Note that the resulting automaton is a PDA.
input
DFA
PDA
AND
accept/reject
stack
52

Complement
Theorem: CFL are not closed under complement.
Proof: by contradiction.
• Take two CFLs L 1 and L 2 and assume that ~L 1
and ~L 2 are CFL. (~L denotes complement)
• CFL are closed under union operation so
~L 1 ~L 2 is a CFL.
• Using our assumption again we get that
~(~L 1 ~L 2 ) is a CFL as well.
• But ~(~L 1 ~L 2 ) =L 1 L 2 and we already know
that CFLs are not closed under intersection.
53