Compile-time Loop Splitting for Distributed Memory ... - Stanford AI Lab

Ten 1-D Loop Nests One 2-D Loop Nest
for(i,0,10,1) � body(i);� for(I step,0,100,10) �
for(i,10,20,1) � body(i);� for(i,I step,I step+10,1)
for(i,20,30,1) � body(i);� � body(i);� �
...
for(i,90,100,1) � body(i);�
Table 3.2: Ten separate loops of equal iteration length can be reduced in code to two nested
loops.
Peeled Iterations Grouped Into Loops
body(i=0);
body(i=1); for(i,0,2,1) � body(i);�
for(i,2,97,1) � body(i);� for(i,2,97,1) � body(i);�
body(i=97); for(i,97,100,1) � body(i);�
body(i=98);
body(i=99);
Table 3.3: The original loop with a prologue and epilogue both peeled away and grouped into
loops themselves.
subloops in entirety. For example, the code on the left of Table 3.2, comprised of ten
subloops of equal length (10 iterations), can be represented by the much smaller code on
the right. With this definition of loop splitting, code expansion is no longer a concern, and
the extra loop overhead is negligible provided the subloops have a moderate iteration size.
3.2.2 Peeling
This thesis defines peeling as the removal of a small number of iterations from either end
of a loop’s iteration interval. These peeled iterations result in a prologue (the first few
iterations) and an epilogue (the last few iterations) around the modified loop. Table 3.3
shows the original loop of Table 3.1 with two iterations peeled into a prologue and three
peeled into an epilogue. Of course, if the number of peeled iterations is large enough, loop
overhead may be favored over code expansion so that the prologue and epilogue become
loops themselves, as Table 3.3 also illustrates.
For the remainder of this thesis, the term loop splitting will refer not to the general
transformation of Section 3.2.1, but specifically to the combination of general loop splitting
and peeling to allow for parallel compiler optimizations, which Section 3.4 describes.
30