Compile-time Loop Splitting for Distributed Memory ... - Stanford AI Lab

3
4 1 5
11 J 12
13
I
19 20 21
2
A B
Figure 2-5: The data footprint of a processor (a) for the expression e‘�“‘�“ ae‘�CI“‘�CI“C
e‘�CI“‘�CP“ and (b) for the more general expression in which ��Cand ��Care the largest
positive offsets for the induction variables � and �, respectively; and ��0 and ��0 are the
smallest negative offsets. The white area represents data for which a value is calculated,
while the shaded areas are the additional data needed for the calculations.
a partitioning groups closely associated iterations on one processor, thereby increasing the
temporal locality by maximizing data reuse. When an iteration needs a particular array
cell, the cell is cached and available to later iterations on the same processor. Because a
network or memory access occurs only once per unique array cell, and because the
suggested tile dimensions minimize the number of different array references; such a task
partition minimizes the total access time and is optimal.
The details of optimal task partitioning are contained in [AKN92], but determining
the optimal aspect ratio for a 2-D loop nest will quickly be presented here.
The derivation of the optimal (to a first approximation) aspect ratio is rather simple.
Finding the I and J resulting in minimal communication we compute their ratio I/J. This is
performed in the following manner.
The tile size is � as 2 t. Communication (to a first approximation) is the number of
rows and columns of nonlocal data. Where is the number of rows and is the number of
columns, the total communication in a multiprocessor with caches is
as Ct as C
�
s
dj-
di-
a �
t CtX
To obtain the I and J that minimize communication, we calculate the derivative of
20
di+
dj+