Programming in Scala”

Chapter notes 20
Notes on chapter 7: Purely functional parallelism
FP has a long history of using combinator libraries for expressing parallelism, and there are a lot
of variations of the general idea. The main design choices in this sort of library are around how
explicit to make the forking and joining of parallel computations. That is, should the API force the
programmer to be fully explicit about when parallel tasks are being forked off into a separate logical
thread, or should this be done automatically? And similarly, when waiting for the results of multiple
logical threads (say, for the implementation of map2), should the order of these joins be something
the programmer explicitly specifies or chosen by the framework?
The library we developed in this chapter sits somewhere in the middle–it is explicit about where
tasks are forked, but not when tasks are joined (notice that map2 picks the order it waits for the two
tasks whose results it is combining). The join order can be made more explicit. Simon Marlow, one
of the GHC Haskell⁸⁶ developers, discusses this in Parallel programming in Haskell with explicit
futures⁸⁷. Also see the full paper, Seq no more: Better Strategies for Parallel Haskell⁸⁸, which does a
nice job of explaining some of the history of approaches for parallelism in Haskell.
Note that because Scala is a strict-by-default language, being more explicit about the join order isn’t
necessarily as beneficial as in Haskell. That is, we can get away with reasoning about join order
much like we think about evaluation in normal strict function application.
This style of library isn’t particularly good at expressing pipeline parallelism that’s possible when
transforming streams of data. For instance, if we have a Stream[Int] of 10000 items, and we wish
to square each value, then compute a running sum of the squared values, there is a potential for
parallelism–as we are squaring values, we can be passing the squared values off to another consumer
that is emitting the running sum of the values it receives. We’ll be discussing stream processing and
pipeline parallelism more in part 4.
Notes about map fusion
We noted in this chapter that one of our laws for Par, sometimes called map fusion, can be used as
an optimization:
map(map(y)(g))(f) == map(y)(f compose g)
That is, rather than spawning a separate parallel computation to compute the second mapping, we
can fold it into the first mapping. We mentioned that our representation of Par doesn’t allow for
this, as it’s too ‘opaque’. If we make Par a proper data type and give it constructors that we can
pattern match on, then it’s easy to implement map fusion:
⁸⁶http://www.haskell.org/ghc/
⁸⁷http://ghcmutterings.wordpress.com/2010/08/20/parallel-programming-in-haskell-with-explicit-futures/
⁸⁸http://www.haskell.org/~simonmar/papers/strategies.pdf