Programming in Scala”

Chapter notes 24
grammars require little lookahead. Interestingly, the grammar ends up looking almost the same with
the introduction of an explicit commit: the leaf-level token parser is usually wrapped in commit, and
higher levels of the grammar that require more lookahead use attempt.
Input handling: Is the input type fixed to be String or can parsers be polymorphic over any sequence
of tokens? The library developed in this chapter chose String for simplicity and speed. Separate
tokenizers are not as commonly used with parser combinators–the main reason to use a separate
tokenizer would be for speed, which is usually addressed by building a few fast low-level primitive
like regex (which checks whether a regular expression matches a prefix of the input string).
Having the input type be an arbitrary Seq[A] results in a library that could in principle be used for
other sorts of computations (for instance, we can “parse” a sum from a Seq[Int]). There are usually
better ways of doing these other sorts of tasks (see the discussion of stream processing in part 4 of
the book) and we have found that there isn’t much advantage in generalizing the library we have
here to other input types.
We can define a similar set of combinators for parsing binary formats. Binary formats rarely have to
deal with lookahead, ambiguity, and backtracking, and have simpler error reporting needs, so often
a more specialized binary parsing library is sufficient. Some examples of binary parsing libraries
are Data.Binary¹⁰⁷ in Haskell, which has inspired similar libraries in other languages, for instance
scodec¹⁰⁸ in Scala.
You may be interested to explore generalizing or adapting the library we wrote enough to handle
binary parsing as well (to get good performance, you may want to try using the specialized
annotation¹⁰⁹ to avoid the overhead of boxing and unboxing that would otherwise occur).
Streaming parsing and push vs. pull: It is possible to make a parser combinator library that can
operate on huge inputs, larger than what is possible or desirable to load fully into memory. Related
to this is is the ability to produce results in a streaming fashion–for instance, as the raw bytes of
an HTTP request are being read, the parse result is being constructed. The two general approaches
here are to allow the parser to pull more input from its input source, and inverting control and
pushing chunks of input to our parsers (this is the approach taken by the popular Haskell library
attoparsec¹¹⁰), which may report they are finished, failed, or requiring more input after each chunk.
We will discuss this more in part 4 when discussing stream processing.
Handling of ambiguity and left-recursion: The parsers developed in this chapter scan input from left
to right and use a left-biased or combinator which selects the “leftmost” parse if the grammar is
ambiguous–these are called LL parsers¹¹¹. This is a design choice. There are other ways to handle
ambiguity in the grammar–in GLR parsing¹¹², roughly, the or combinator explores both branches
simultaneously and the parser can return multiple results if the input is ambiguous. (See Daniel
¹⁰⁷http://code.haskell.org/binary/
¹⁰⁸https://github.com/scodec/scodec
¹⁰⁹http://www.scala-notes.org/2011/04/specializing-for-primitive-types/
¹¹⁰http://hackage.haskell.org/packages/archive/attoparsec/0.10.2.0/doc/html/Data-Attoparsec-ByteString.html
¹¹¹http://en.wikipedia.org/wiki/LL_parser
¹¹²http://en.wikipedia.org/wiki/GLR_parser