Programming in Scala”

Chapter notes 22
forAll(Gen.listOf(Gen.choose(1,10))) { l =>
val buf = new collection.mutable.ArrayStack[Int]
val beforeSize = buf.size
l.foreach(buf.push)
buf.beforeSize == 0 && buf.size == l.size
}
In this case, the “interpreter” is the push method on ArrayStack, which modifies the stack in place,
and the “instructions” are simply the integers from the input list. But the basic idea can be extended
to testing richer interfaces–for instance, we could generate instructions that could either push or pop
elements from an ArrayStack (perhaps represented as a List[Option[Int]]), and write a property
that sequences of push and pop preserve the invariants of ArrayStack (for instance, the final size
of the stack should be the number of push calls minus the number of pop calls). Care must be
taken to craft generators that produce valid sequences of instructions (for instance, pop without a
corresponding prior push is not a valid input).
Similar ideas have been used for testing thread safety of concurrent programs. (See Finding Race
Conditions in Erlang with QuickCheck and PULSE⁹⁹ (PDF)) The key insight here is that threadsafe
code does not allow the nondeterminism of thread scheduling to be observable. That is, for
any partial order of instructions run concurrently, we ought to able to find some single-threaded
linear sequence of these instructions with the same observable behavior (this criteria is often called
linearizability). For instance, if our ArrayStack were thread-safe, we would expect that if 2 push
operations were performed sequentially, followed by two pop operations and two push operations
performed concurrently, this should yield the same result as some deterministic linear sequence of
these push and pop operations). There are some subtleties and interesting questions about how to
model this and how to report and minimize failing test cases. In particular, doing it the “obvious” way
ends up being intractable due to having to search through a combinatorial number of interleavings to
find one that satisfies the observation. The Erlang paper linked above has more details, in particular
see section 4. You may be interested to explore how to incorporate these ideas into the library we
developed, possibly building on the parallelism library we wrote last chapter.
Lastly, we mention that one design goal of some libraries in this style is to avoid having to explicitly
construct generators. The QuickCheck library makes use of a Haskell type class to provide instances
of Gen “automatically”, and this idea has also been borrowed by ScalaCheck¹⁰⁰. This can certainly
be convenient, especially for simple examples, though we often find that explicit generators are
necessary to capture all the interesting constraints on the shape or form of the inputs to a function.
⁹⁹http://www.protest-project.eu/upload/paper/icfp070-claessen.pdf
¹⁰⁰https://github.com/rickynils/scalacheck/wiki/User-Guide