Recursive descent parsing for grammars with contexts

39th International Conference on Current Trendsin Theory and Practice of Computer ScienceŠpindleruv Mlýn, Czech RepublicRecursive descent parsingfor grammars with contextsMikhail BarashPh.D. student,Department of Mathematics and Statistics, University of TurkuTurku Centre for Computer Science (TUCS)FinlandJanuary 26–31, 2013Mikhail BarashRecursive descent for grammars with contexts

What a recursive descent is?subclass of grammars allowing recursive descent:LL(k)-grammarsk look-ahead symbolsMikhail BarashRecursive descent for grammars with contexts

What a recursive descent is?subclass of grammars allowing recursive descent:LL(k)-grammarsk look-ahead symbolsLL(1) context-free grammar generating { a n b n | n 0 }:S → aSb | ε★S() { a() {if (look-ahead is “a”) { if (current symbol is “a”)a(); S(); b(); advance position by 1} else error} }✧✥✦Mikhail BarashRecursive descent for grammars with contexts

What a recursive descent is?subclass of grammars allowing recursive descent:LL(k)-grammarsk look-ahead symbolsLL(1) context-free grammar generating { a n b n | n 0 }:S → aSb | ε★S() { a() {if (look-ahead is “a”) { if (current symbol is “a”)a(); S(); b(); advance position by 1} else error} }✧First compilers for Pascal are recursive descent parsers.Implemented “by hand” (N. Wirth, 1970).Program code can be easily generated automatically.✥✦Mikhail BarashRecursive descent for grammars with contexts

Extension of CFGs with Boolean operationsConjunctive grammars (Okhotin, 2001)A → α 1 & . . . & α k ,α i ∈ (Σ ∪ N) ∗Mikhail BarashRecursive descent for grammars with contexts

Extension of CFGs with Boolean operationsConjunctive grammars (Okhotin, 2001)A → α 1 & . . . & α k , α i ∈ (Σ ∪ N) ∗“a string w has property A ⇐⇒ w has all the properties α 1, . . . , α k ”Mikhail BarashRecursive descent for grammars with contexts

Extension of CFGs with Boolean operationsConjunctive grammars (Okhotin, 2001)A → α 1 & . . . & α k ,α i ∈ (Σ ∪ N) ∗“a string w has property A ⇐⇒ w has all the properties α 1, . . . , α k ”{ a n b n c n | n 0 }S → AB & DCA → aA | ε B → bBc | εC → cC | ε D → aDb | εMikhail BarashRecursive descent for grammars with contexts

Extension of CFGs with Boolean operationsConjunctive grammars (Okhotin, 2001)A → α 1 & . . . & α k ,α i ∈ (Σ ∪ N) ∗“a string w has property A ⇐⇒ w has all the properties α 1, . . . , α k ”{ a n b n c n | n 0 }S → AB & DCA → aA | ε B → bBc | εC → cC | ε D → aDb | εnon-context-free languages generatedcomplexity of basic parsing algorithms preservednontrivial properties of subclassesMikhail BarashRecursive descent for grammars with contexts

Grammars with contexts (B, Okhotin, 2012)A → α 1 & . . . & α kMikhail Barash Recursive descent for grammars with contexts

Grammars with contexts (B, Okhotin, 2012)A → α 1 & . . . & α k & ✄β 1 & . . . & ✄β mMikhail BarashRecursive descent for grammars with contexts

Grammars with contexts (B, Okhotin, 2012)A → α 1 & . . . & α k & ✄β 1 & . . . & ✄β m & γ 1 & . . . & γ nMikhail BarashRecursive descent for grammars with contexts

Grammars with contexts (B, Okhotin, 2012)A → α 1 & . . . & α k & ✄β 1 & . . . & ✄β m & γ 1 & . . . & γ nGiven a string x = u w v:each α i defines wMikhail BarashRecursive descent for grammars with contexts

Grammars with contexts (B, Okhotin, 2012)A → α 1 & . . . & α k & ✄β 1 & . . . & ✄β m & γ 1 & . . . & γ nGiven a string x = u w v:each α i defines weach β i defines v (the right context of w)Mikhail BarashRecursive descent for grammars with contexts

Grammars with contexts (B, Okhotin, 2012)A → α 1 & . . . & α k & ✄β 1 & . . . & ✄β m & γ 1 & . . . & γ nGiven a string x = u w v:each α i defines weach β i defines v (the right context of w)each γ i defines wv (the extended right context of w)Mikhail BarashRecursive descent for grammars with contexts

Grammars with contexts (B, Okhotin, 2012)A → α 1 & . . . & α k & ✄β 1 & . . . & ✄β m & γ 1 & . . . & γ nGiven a string x = u w v:each α i defines weach β i defines v (the right context of w)each γ i defines wv (the extended right context of w)Semantics by logical deduction of elementary propositions[X , 〈w〉v] : “a string w written in right context v has the property X ”Mikhail BarashRecursive descent for grammars with contexts

Grammars with contexts (B, Okhotin, 2012)A → α 1 & . . . & α k & ✄β 1 & . . . & ✄β m & γ 1 & . . . & γ nGiven a string x = u w v:each α i defines weach β i defines v (the right context of w)each γ i defines wv (the extended right context of w)Semantics by logical deduction of elementary propositions[X , 〈w〉v] : “a string w written in right context v has the property X ”{a n b n c} ∪ {a n b 2n d}Mikhail BarashRecursive descent for grammars with contexts

Grammars with contexts (B, Okhotin, 2012)A → α 1 & . . . & α k & ✄β 1 & . . . & ✄β m & γ 1 & . . . & γ nGiven a string x = u w v:each α i defines weach β i defines v (the right context of w)each γ i defines wv (the extended right context of w)Semantics by logical deduction of elementary propositions[X , 〈w〉v] : “a string w written in right context v has the property X ”{a n b n c} ∪ {a n b 2n d}S → Ac | BdA → aAb | ε L(A) = {a n b n }B → aBbb | ε L(B) = {a n b 2n }Mikhail BarashRecursive descent for grammars with contexts

Grammars with contexts (B, Okhotin, 2012)A → α 1 & . . . & α k & ✄β 1 & . . . & ✄β m & γ 1 & . . . & γ nGiven a string x = u w v:each α i defines weach β i defines v (the right context of w)each γ i defines wv (the extended right context of w)Semantics by logical deduction of elementary propositions[X , 〈w〉v] : “a string w written in right context v has the property X ”{a n b n c} ∪ {a n b 2n d}S → Ac | BdA → aAb | ε L(A) = {a n b n }B → aBbb | ε L(B) = {a n b 2n }the language is not standard LL(k) for any kMikhail BarashRecursive descent for grammars with contexts

Grammars with contexts (B, Okhotin, 2012)A → α 1 & . . . & α k & ✄β 1 & . . . & ✄β m & γ 1 & . . . & γ nGiven a string x = u w v:each α i defines weach β i defines v (the right context of w)each γ i defines wv (the extended right context of w)Semantics by logical deduction of elementary propositions[X , 〈w〉v] : “a string w written in right context v has the property X ”{a n b n c} ∪ {a n b 2n d}S → Ac | BdA → aAb | ε L(A) = {a n b n }B → aBbb | ε L(B) = {a n b 2n }the language is not standard LL(k) for any kway out: use right contexts to peek the last symbol of the stringMikhail BarashRecursive descent for grammars with contexts

Grammars with contexts (B, Okhotin, 2012)A → α 1 & . . . & α k & ✄β 1 & . . . & ✄β m & γ 1 & . . . & γ nGiven a string x = u w v:each α i defines weach β i defines v (the right context of w)each γ i defines wv (the extended right context of w)Semantics by logical deduction of elementary propositions[X , 〈w〉v] : “a string w written in right context v has the property X ”{a n b n c} ∪ {a n b 2n d}S → Ac | BdA → aAb | ε L(A) = {a n b n }B → aBbb | ε L(B) = {a n b 2n }X → aX | bX | ε L(X ) = (a | b) ∗the language is not standard LL(k) for any kway out: use right contexts to peek the last symbol of the stringMikhail BarashRecursive descent for grammars with contexts

ResultsRecursive descent parser:Mikhail BarashRecursive descent for grammars with contexts

ResultsRecursive descent parser:conjunction in rules: scan the substring multiple timesMikhail BarashRecursive descent for grammars with contexts

ResultsRecursive descent parser:conjunction in rules: scan the substring multiple timesright contexts (✄, ): to choose suitable alternatives of rulesMikhail BarashRecursive descent for grammars with contexts

ResultsRecursive descent parser:conjunction in rules: scan the substring multiple timesright contexts (✄, ): to choose suitable alternatives of rulesbacktracking: to go over the alternativesMikhail BarashRecursive descent for grammars with contexts

ResultsRecursive descent parser:conjunction in rules: scan the substring multiple timesright contexts (✄, ): to choose suitable alternatives of rulesbacktracking: to go over the alternativesmemoization: linear time instead of exponentialMikhail BarashRecursive descent for grammars with contexts

ResultsRecursive descent parser:Theory:conjunction in rules: scan the substring multiple timesright contexts (✄, ): to choose suitable alternatives of rulesbacktracking: to go over the alternativesmemoization: linear time instead of exponentialmathematically sound definition of right contexts(“look-ahead”) within recursive descentMikhail BarashRecursive descent for grammars with contexts

ResultsRecursive descent parser:Theory:conjunction in rules: scan the substring multiple timesright contexts (✄, ): to choose suitable alternatives of rulesbacktracking: to go over the alternativesmemoization: linear time instead of exponentialmathematically sound definition of right contexts(“look-ahead”) within recursive descenthigher expressive power, but still linear timeMikhail BarashRecursive descent for grammars with contexts

ResultsRecursive descent parser:Theory:conjunction in rules: scan the substring multiple timesright contexts (✄, ): to choose suitable alternatives of rulesbacktracking: to go over the alternativesmemoization: linear time instead of exponentialmathematically sound definition of right contexts(“look-ahead”) within recursive descenthigher expressive power, but still linear timeparsing algorithm proven correctMikhail BarashRecursive descent for grammars with contexts

ResultsRecursive descent parser:Theory:conjunction in rules: scan the substring multiple timesright contexts (✄, ): to choose suitable alternatives of rulesbacktracking: to go over the alternativesmemoization: linear time instead of exponentialmathematically sound definition of right contexts(“look-ahead”) within recursive descentPractice:higher expressive power, but still linear timeparsing algorithm proven correctimplemented as a prototype softwareMikhail BarashRecursive descent for grammars with contexts

ResultsRecursive descent parser:Theory:conjunction in rules: scan the substring multiple timesright contexts (✄, ): to choose suitable alternatives of rulesbacktracking: to go over the alternativesmemoization: linear time instead of exponentialmathematically sound definition of right contexts(“look-ahead”) within recursive descentPractice:higher expressive power, but still linear timeparsing algorithm proven correctimplemented as a prototype softwareMikhail BarashRecursive descent for grammars with contexts