U. Glaeser

synchronization) and usually contain multiple execution units (EUs). The PEs are interconnected by some
network or through centralized resources such as register file and memory, for inter-PE communication.
Our definition of a PE is somewhat loose. On one extreme, the PEs in some multithreaded processors
are separate processor-memory systems with their own instruction cache, decode unit, register file, and
execution units; on the other extreme, the PEs in some multithreaded processors even share the execution
units, as in the dynamic multithreading processor [1]. Such a loose definition allows us to discuss a wide
spectrum of multithreaded processors under a common framework.
Number of PEs and PE Organization
The number of PEs in a multiprocessor is an important hardware parameter. This number is strongly
tied to the perceived parallelism in the targeted application domain, and also the nature of the threads.
On one extreme, we have single-PE multithreaded processors that perform time sharing. On the other
extreme, we have massively parallel processors (MPPs) consisting of thousands of PEs, which are the
most powerful machines available today for many time-critical applications [4]. Because of the sharp
increase in the number of transistors integrated in a single chip, there is significant interest in integrating
multiple PEs in the same chip. This has been the motivation behind many of the SpMT processing
models.
Processor Context Interleaving
When the number of parallel threads exceeds the number of PEs, it is possible to time-share a single PE
among multiple threads in a way that minimizes the time required to switch threads. This is accomplished
by sharing as much as possible of the program execution environment between the different threads so
that very little state needs to be saved and restored when changing threads. This type of low-overhead
interleaving is given the name multithreading in many circles [2,3,17]. Interleaving-based multithreading
differs from conventional multitasking (or multiprogramming) in that the concurrent threads share more
of their environment with each other than do concurrent tasks under multitasking. Threads may be
distinguished only by the value of their program counters and stack pointers while sharing a single address
space and set of global variables. As a result, there is very little protection of one thread from another,
in contrast to multitasking. Interleaving-based multithreading can thus be used for very fine-grain
multitasking, at the level of a few instructions, and so can hide latency by keeping the processor busy
after one thread issues a long-latency instruction on which subsequent instructions in that thread depend.
• Cycle-level interleaving: In this scheme, a PE switches to a different thread after each instruction
fetch; i.e., an instruction of another thread is fetched and fed into the execution pipeline in the
next clock cycle. Cycle-level interleaving is typically used for coarse-grain threads—processes or
light-weight processes. The motivation for this is that it eliminates control and data dependences
between the instructions that are simultaneously active in the pipeline. Thus, there is no need to
build complex forwarding paths, permitting a simple and potentially fast pipeline. Furthermore,
the context switch latency is zero cycles. Memory latency is tolerated by not scheduling a thread
until the memory access has been completed. For this interleaving to work well, there must be as
many threads as the worst-case latencies experienced by the instructions. Interleaving the instructions
from many threads limits the processing speed of a single thread, thereby degrading single-thread
performance. The most well-known examples of cycle-level interleaving processors are HEP [29],
Horizon [33], and Tera MTA [2].
• Block interleaving: In this scheme, the instructions of a thread are executed successively until a
long-latency event occurs, which causes a context switch. A typical long-latency operation is a
remote memory access. Compared to the cycle-level interleaving technique, a smaller number of
threads is sufficient, and a single thread can execute at full speed until the next context switch.
The events that cause a context switch can be determined statically or dynamically.
When hardware technology allows more PEs to be integrated in a processor, PE interleaving becomes
less attractive, because computational throughput will clearly improve when multiple threads execute in
© 2002 by CRC Press LLC