Compile-time Loop Splitting for Distributed Memory ... - Stanford AI Lab

For example, in the context of the previous 2-D loop nest, a task partition with
blocking (Figure 2-3b) and a data partition with striping by rows (Figure 2-4b) have poor
alignment. Even with optimal placement (virtual processors 0, 1, 2, 3 map to real
processors 3, 2, 1, 0, respectively), only half of a processor’s data footprint is in its local
memory. Processor Zero (with virtual PID 3) would require data in both local memory and
the memory of Processor One (with virtual PID 2). For Processor Zero to reference the
data, it must not only specify the address of an array cell but also the processor’s memory
in which it is located.
Thus, the very definition of a distributed memory multiprocessor leads to
complications in referencing the program’s dispersed data. The complication appears in the
form of involved reference expressions, which adversely affect the execution time of a
program. The next section describes these expressions and illustrates the possibilities for
simplification.
2.5 Array Referencing
Methods of array referencing on distributed memory multiprocessors vary; the method
described here is used by the Alewife effort ([Aet al.91]) at MIT. This thesis concerns itself
with rectangular task and data partitioning and barrier synchronization of parallel loop
nests, both of which are provided by the Alewife compiler.
Array referencing in Alewife is implemented in software. The general expression to
access an array cell is
—��� @—��� @—��—�Y ���AY ������AY
where —��� is the array reference procedure, —��—� is the name of the array, ���is the
unique ID of the processor whose memory contains the cell, and ������ is the offset into that
memory. The reference procedure —��� has two arguments – a list structure and an offset
into the structure, and returns the element in the structure located at the offset. In the
general expression above, the inner —��� determines the memory segment in which the cell
resides from both —��—�’s dope vector (a list of pointers to memory segments) and the
processor ID. The outer —��� uses the memory segment and ������ to obtain the actual
22