Architecture of Computing Systems (Lecture Notes in Computer ...

238 F. Xudong et al.
stencil computations of Mgrid, many intermediate results can be reused. Reusing
these results reduces memory fetches as well as computation. Generally, large
thread granularity can improve data locality and computation intensity, though
entailing the consumption of more GPRs. We will explain the concept of thread
granularity later.
Given limited resources (such as the number of GPRs, memory bandwidth)
and the amount of work (e.g., a specific kernel computation), the number of
threads that can be created is determined bythreadgranularity.Inotherwords,
the thread granularity is in inverse proportion to the number of threads. This
means tuning thread granularity can balance locality within thread and parallelism
among threads [8]. So there should be a balance between thread locality
and thread parallelism through tuning. To achieve the best performance, programmers
should carefully tune the thread granularity to strike the right balance.
In Brook+, the total number of threads is determined by the output stream
size (domain size). Note that the thread granularity here means the number
of grid points a thread calculates. There are two methods to tuning thread
granularity in Brook+:
(a) Using vector types
Brook+ provides built-in short vector types for tuning the code explicitly on
the available short-SIMD machines. Short vectors here are built from the name of
their base type, with the size appended as a suffix, such as float4 and double2.
Using vector types reduces the domain size (the output stream length) by a
factor of the vector size, and consequently increases the thread granularity by
the same factor. Take double2 for example. Using double2 increases the thread
granularity by a factor of two. A thread can now compute two stencil points at a
time, so more data reuse can be exploited through using the intermediate results
within each thread.
Moreover, using vector types can pack up to four scalar fetches into one vector
fetch to form a vector fetch. Vector fetch significantly reduces memory fetches
if a kernel is designed to fetch from consecutive data locations, thus making
more efficient use of the fetch resources. For example, a kernel can issue a float4
fetch in one cycle versus four separate float fetches in four cycles. In the stencil
computations of Resid where locations of the data to be fetched are usually consecutive,
vector fetches naturally raise arithmetic intensity and improve memory
performance.
(b) Multiple output streams
Brook+ supports up to eight simultaneous output streams per kernel using the
CAL backend [1]. Using multiple output streams, a thread can complete multifolds
of computations with respect to using a single output stream, which also
increases the thread granularity. For example, using two output streams doubles
the thread granularity. If we combine vector types and multiple output streams
together, even larger thread granularity can be attained. For instance, using
double2 and four output streams together, we increase the thread granularity by
a factor of eight (24=8).