Core Python Programming (2nd Edition)

11.10. Generators
Earlier in Chapter 8, we discussed the usefulness behind iterators and how they give non-sequence
objects a sequence-like iteration interface. They are simple to understand because they only have one
method, a next() that is called to get the next item.
However, unless you implement a class as an iterator, iterators really do not have much "intelligence."
Would it not be much more powerful to call a function that somehow "generated" the next value in the
iteration and returned with something as simple as a next() call? That is one motivation for generators.
Another aspect of generators is even more powerful ... the concept of coroutines. A coroutine is an
independent function call that can run, be paused or suspended, and be continued or resumed where it
left off. There is also communication between the caller and the (called) coroutine. For example, when a
coroutine pauses, we can receive an intermediate return value from it, and when calling back into one,
to be able to pass in additional or altered parameters, yet still be able to pick up where we last left it,
with all state still intact.
Coroutines that are suspended yielding intermediate values and resumed multiple times are called
generators, and that is exactly what Python generators do. Generators were added to Python in 2.2 and
made standard in 2.3 (see PEP 255), and although powerful enough, they were significantly enhanced in
Python 2.5 (see PEP 342). These enhancements bring generators even closer to being full coroutines
because values (and exceptions) are allowed to be passed back into a resumed function. Also,
generators can now yield control while waiting for a generator it has called to yield a result instead of
blocking to wait for that result to come back before the calling generator can suspend (and yield a
result). Let us take a closer look at generators starting from the top.
What is a generator Python-wise? Syntactically, a generator is a function with a yield statement. A
function or subroutine only returns once, but a generator can pause execution and yield intermediate
resultsthat is the functionality of the yield statement, to return a value to the caller and to pause
execution. When the next() method of a generator is invoked, it resumes right where it left off (when it
yielded [a value and] control back to the caller).
When generators were added back in 2.2, because it introduced a new keyword, yield, for backward
compatibility, you needed to import generators from the __future__ module in order to use them. This
was no longer necessary when generators became standard beginning with 2.3.
11.10.1. Simple Generator Features
Generators behave in another manner similar to iterators: when a real return or end-of-function is
reached and there are no more values to yield (when calling next()), a StopIteration exception is
raised. Here is an example, the simplest of generators:
def simpleGen():
yield 1
yield '2 --> punch!'