Top-down Parsing (ASU Ch 4.4) Predictive Parsers (ASU 4.4 ...

Top-down Parsing (ASU Ch 4.4)
• Predictive Parser (PP) (non-backtracking)
• define class of LL(1) - PP can be generated automatically
• top-down parsing
– recursive descent (RD)
– non-recursive descent (NRD)
• backtracking example (ASU Fig. 4.9)
S cAd A ab | a w = cad (input string)
S cAd cabd
(no match - backtrack!)
S cAd cad
(match - OK)
exercise: draw the parse trees (or see ASU Fig 4.9)
1
14/10/2014 DFR - CC - TDP

Predictive Parsers (ASU 4.4)
• Predictive Parsers (PP) (no backtracking)
– eliminate left recursion
– left factor the resulting grammar
– implementation: recursive descent (RDPP)
• Transition Diagrams for PPs
– for each NT in grammar G
• create an initial (IS) and final state (FS)
• for each P: A X 1 X 2 X 3 …X n create a path from IS to FS with
edges labelled X 1 X 2 X 3 … X n
• see ASU pp 183-185 for examples (Figs 4.10, 4.11, 4.12)
2
14/10/2014 DFR - CC - TDP

Transition Diagrams (ASU pp183-185)
• Transition diagram behaviour
– begin with IS for start symbol S (in G)
– for state s a t
• if input (from w) is a (a is T), read input symbol & go to state t
– for state s A t
• A is NT, go to IS for A
• effectively A has been read in from the input stream (w)
• on FS for A, go to state t
– for state s ¤ t (¤ = empty)
• go to state t (do not read from the input stream)
– implementation
• match Ts against the input stream (w)
• for an NT, call a procedure (possibly recursively)
3
14/10/2014 DFR - CC - TDP

Transition Diagrams: example (ASU pp183-185)
E TE’
E’ +TE’ | ¤
T FT’
T’ *FT’ | ¤
F (E) | id
T E’
0 1 2
3
+
4
T
5
E’
6
¤
7
F
8
T’
9
10
*
11
F
12
T’
13
¤
14
(
15
E
16
)
17
id
4
14/10/2014 DFR - CC - TDP

Recursive Descent example
L ::= E { ; E } ’;’ ’\n’
E ::= E ’+’ T | T
T ::= T ’*’ F | F
E
L
;
\n
F ::= ’(’ E ’)’ | digit
right recursive
T
E’
L ::= E { ; E } ; ’\n’
E ::= T E’
F
T’
+ T
E’
E’ ::= ¤ | ’+’ T E’
T ::= F T’
F T’
T’ ::= ¤ | ’*’ F T’
F ::= ’(’ E ’)’ | digit
2 e + 2
e
e
;
\n
5
14/10/2014 DFR - CC - TDP

Transition Diagrams: example (ASU pp183-185)
• Optimisation
E E+T | T
T T*F | F
F (E) | id
+
0
T
3
¤
6
*
F ¤
7 8 13
14
(
15
E
16
)
17
id
6
14/10/2014 DFR - CC - TDP

Non Recursive Predictive Parser (NRPP) (ASU
Ch 4.4, Fig 4.13)
• table driven + stack
– $ : end-of-input / bos
– M[A, a] - A NT, a T or $
– X is symbol on tos
• actions
– X = a = $ : halt, success
– X = a != $ : pop X,
advance input pointer
– X is NT : see M[X, a]
stack
X
Y
Z
$
• X => UVW (expand X on stack)
• X is error - call error recovery
input (w)
a + b $
PP program
parsing table
M
output
7
14/10/2014 DFR - CC - TDP

First and Follow Sets (ASU Ch 4.4)
• first(X)
– if X is T first(X) is { X }
– if X ¤ in P first(X) += ¤ (¤ = empty)
– if X is NT and X Y 1 Y 2 Y 3 …Y n in P
• first(X) += a if a in first(Y i ) and ¤ in first(Y 1 ) … first(Y i-1 )
– i.e. Y 1 Y 2 Y 3 …Y i-1 ¤
– if ¤ in first(Y 1 ) … first(Y k ) first(X) += ¤
– first(X) += first(Y 1 ) if not Y 1 ¤
– first(X) += first(Y 2 ) if Y 1 ¤ and not Y 2 ¤
– and so on ...
– continued on next slide
8
14/10/2014 DFR - CC - TDP

First and Follow Sets (ASU Ch 4.4)
• first(X) continued (the implications of ¤)
– for any string X 1 X 2 X 3 … X n
• first( X 1 X 2 X 3 … X n ) += non ¤ symbols in first( X 1 )
• if ¤ in first( X 1 )
– first( X 1 X 2 X 3 … X n ) += non ¤ symbols in first( X 2 )
• if ¤ in first( X 1 ) and ¤ in first( X 2 )
– first( X 1 X 2 X 3 … X n ) += non ¤ symbols in first( X 3 )
– and so on
• if ¤ in first( X i ) i = 1..n, first( X 1 X 2 X 3 … X n ) += ¤
9
14/10/2014 DFR - CC - TDP

First: example (ASU Ex 4.17)
E TE’
E’ +TE’ | ¤
T FT’
T’ *FT’ | ¤
F (E) | id
• first(E) = first(T) = first(F) = {(, id}
– from F (E) | id;
T FT’;
E TE’
• first(E’) = { +, ¤ }
– from E’ +TE’ | ¤
• + is a terminal
• E’ ¤
• first(T’) = { *, ¤ }
– from T’ *FT’ | ¤
• * is a terminal
• T’ ¤
10
14/10/2014 DFR - CC - TDP

First and Follow Sets (ASU Ch 4.4)
• follow(X)
– follow(S) += $ S: start symbol / $: end of input / S = w$
– if A => a B β sibling
• follow(B) += first(β) except for ¤
– if in P : A => aB or A => aBβ and ¤ in first (β)
• follow(B) += follow(A)
(¤=empty)
parent
11
14/10/2014 DFR - CC - TDP

Follow: example (ASU Ch 4.4)
E TE’
E’ +TE’ | ¤
T FT’
T’ *FT’ | ¤
F (E) | id
first(E) = { (, id }
first (E’) = { +, ¤ }
first(T) = { (, id }
first (T’) = { *, ¤ }
first(F) = { (, id }
follow(E) = { ), $ } F (E) and E is S
follow(E’) = { ), $ } E TE’; rule A aB & f(E’) += f(E)
follow(T) = { +, ), $ } E’+TE’|¤ f(T) += f(E’); ETE’ f(T) +=first(E’) -¤
follow(T’) = { +, ), $ } TFT’ & rule AaB f(T’) += f(T)
follow(F) = { +, *, ), $ } TFT’ f(F) += f(T); T’*FT’ f(F) += first(T’) -¤
12
14/10/2014 DFR - CC - TDP

Parsing Table M (ASU Fig 4.15 p188)
NTs id + * ( ) $
E
E’
T
T’
F
ETE’
TFT’
Fid
error state
E’+TE’
T’¤
Input Symbol (Ts)
T’*FT’
ETE’
TFT’
F(E)
E’¤
T’¤
E’¤
T’¤
13
14/10/2014 DFR - CC - TDP

Constructing the Parsing Table
• principles
– if B β in P and b in first(β)
• expand B by β when the current input symbol is b
– if β ¤ or β * ¤
• expand B by β if the input symbol is in follow(Β)
• or $ reached on input and $ in follow(β)
• construction
– for each P: B => β do
• for each T b in first(β); M[B, b] += B β
• if ¤ in first(β) M[B, c] += B β for each T c in follow(Β)
• if ¤ in first(β) and $ in follow(β) M[B, $] += B β
• make each undefined entry in M an error
14
14/10/2014 DFR - CC - TDP

Constructing the Parsing Table: Details
E TE’ add ETE’ to M[E, id] id in first(TE’)
add ETE’ to M[E, ( ]
( in first(TE’)
E’ +TE’ | ¤ add E’+TE’ to M[E’, +] + in first(+TE’)
add E’ ¤ to M[E’, ) ]
¤ in first(E’)
) in follow(E’)
add E’ ¤ to M[E’, $ ]
¤ in first(E’)
$ in follow(E’)
T FT’ add TFT’ to M[E, id] id in first(FT’)
add TFT’ to M[E, ( ]
( in first(FT’)
15
first: E = { (, id } E’ = { +, ¤} T ={ (, id } T’ = { *, ¤} F ={ (, id }
follow: E = { ), $ } E’ = { ), $ } T = { +, ), $ } T’ = { +, ), $ } F = { +, *, ), $ }
14/10/2014 DFR - CC - TDP

Constructing the Parsing Table: Details
T’ *FT’ | ¤ add T’*FT’ to M[T’, *] * in first(*FT’)
add T’ ¤ to M[T’, + ]
¤ in first(T’)
+ in follow(T’)
add T’ ¤ to M[T’, ) ]
¤ in first(T’)
) in follow(T’)
add T’ ¤ to M[T’, $ ]
¤ in first(T’)
$ in follow(T’)
F (E) | id add F (E) to M[F, ( ] ( in first((E))
add F id to M[E, id ]
id in first(id)
first: E = { (, id } E’ = { +, ¤} T ={ (, id } T’ = { *, ¤} F ={ (, id }
follow: E = { ), $ } E’ = { ), $ } T = { +, ), $ } T’ = { +, ), $ } F = { +, *, ), $ }
16
14/10/2014 DFR - CC - TDP

Non Recursive Parsing Algorithm
(ASU Fig 4.14)
input: string w, parsing table M for grammar G
output: if w in L(G), leftmost derivation of w, else error
init: stack $S i/p buffer w$
ip first_symbol(w)
do {
//ip - input pointer, X is TOS
if ( X is T or $) { if (X==*ip) { pop X; ip++} else error( ) } // match
else if ( M[X, *ip] == P: X => Y 1 Y 2 …Y k )
{ pop X; push Y k Y k-1 …Y 1 on to stack; output P }
else error( )
} while ( X != $)
// expand
17
14/10/2014 DFR - CC - TDP

Parse of id+id*id (ASU Fig 4.16)
stack i/p o/p
$E id+id*id$
$E’T id+id*id$ E=>TE’
$E’T’F id+id*id$ T=>FT’
$E’T’id id+id*id$ F=>id
$E’T’ +id*id$
$E’ +id*id$ T’=>¤
$E’T+ +id*id$ E’=>+TE’
$E’T id*id$
$E’T’F id*id$ T=>FT’
$E’T’id id*id$ F=>id
stack i/p
o/p
$E’T’ *id$
$E’T’F* *id$
$E’T’F id$
$E’T’id id$ F=>id
$E’T’ $
$E’ $ T’=>¤
$ $ E’=>¤
finish - success!
T’=>*FT’
18
14/10/2014 DFR - CC - TDP

Properties of LL(1) Grammars (ASU Ch 4.4)
• Left recursive or ambiguous G cannot be LL(1)
• G is LL(1) iff whenever A=> α | β are 2 distinct Ps of G and
1. for no T a do both α β derive strings beginning with a
2. at most one of α β can derive ¤ (¤ = empty)
3. if β =*=> ¤ then α does not derive any string beginning with a
T in follow(A)
• use left recursion elimination / left factoring to produce LL(1) G
• PP used for control constructs / operator precedence for exprs
• error recovery in PP
– Ts, NTs made explicit in stack in PP
– error => TOS T != a (next input symbol) or M[A, a] is empty
– synchronising set (SySt) - panic recovery for NT A SySt += follow(A)
– SySt += keywords
– if TOS is unmatched T, pop, print message and continue
19
14/10/2014 DFR - CC - TDP

Summary
• Top-down parsing
– predictive parser (PP)
– recursive descent (RD)
– non recursive descent (NRD)
– avoid backtracking
• Transition Diagrams for PPS
– example vs RD example
• Sets First and Follow
– derivation & use
• Non Recursive Descent
(NRD)
– Parsing Table M
• principles
• construction
– NRD algorithm
– example parse id+id*id
– properties of LL(1) Gs
• error recovery for PPs
20
14/10/2014 DFR - CC - TDP