U. Glaeser

Studies of Instruction-Level Parallelism
In the early 1970s, two studies on decoding and executing multiple instructions per cycle were published—
one by Gary Tjaden and Mike Flynn on a design of a multiple-issue IBM 7094 [2] and the other by
Ed Riseman and Caxton Foster on the effect of branches in CDC 3600 programs [3]. The conclusion in both
papers was that only a small amount of instruction-level parallelism existed in sequential programs—1.86
and 1.72 instructions per cycle determined by the respective studies. Thus, these studies clearly demonstrated
the limiting effect of data and control dependencies on instruction-level parallelism, and the result was to
encourage researchers to look for parallelism in other arenas, such as vector processors and multiprocessors.
However, the Riseman and Foster study did examine the effect of relaxing the control dependencies and
found increasing levels of parallelism, up to 51 instructions per cycle, as the number of branches were
eliminated (albeit in an impractical way). Later studies, in which false data dependencies were eliminated
as well as control dependencies, found much more available parallelism, with the highest published estimate
being 90 instructions per cycle by Alexandru Nicolau and Josh Fisher as part of their VLIW research [4].
Techniques to Increase Instruction-Level Parallelism
Just as the limit studies indicated, performance can be increased if dependencies can be eliminated or
reduced. Let us address the dependencies in the reverse order from their enumeration above. First, many
structural dependencies can be avoided by providing duplicate copies of necessary resources. Even scalar
pipelines provide two paths for memory access (i.e., separate instruction and data caches) and multiple
adders (i.e., branch target adder and main ALU). Superscalar processors have even more resource requirements,
and it is not unusual to find duplicated function units and even multiple ports to the data
cache (e.g., true multiporting, multiple banks, or accessing a single-ported cache multiple times per cycle).
Control dependencies are eliminated by compiler techniques of unrolling loops and performing
optimizations such as “if conversion” (i.e., using conditional or predicated execution of instructions so
that a control-dependent instruction is transformed into a data-dependent instruction). However, the
main approach to reducing the impact of control dependencies is the use of sophisticated branch
prediction. For example, the Pentium 4 keeps the history of over 4000 branches [5]. Branch prediction
techniques allow instructions from the predicted path to begin before the branch is resolved and execute
in a speculative manner. Of course, if a prediction is incorrect, there must be a way to recover and restart
execution along the correct path.
False data dependencies can be eliminated or reduced by better compiler techniques (e.g., register and
memory allocation algorithms that avoid reuse) or by the use of register and memory renaming hardware
on the processor. Register renaming can be accomplished in the hardware by incorporating a larger set
of physical registers than are available in the instruction set architecture. Thus, as each instruction is
decoded, that instruction’s architectural destination register is mapped to a new physical register, and
future uses of that architectural register will be mapped to the assigned physical register. Hardware
renaming is especially important for older instruction sets that have few architectural registers and for
legacy codes that for one reason or another will not be recompiled.
True data dependencies have been viewed as the fundamental limit for program execution; however,
value prediction has been proposed in the past few years as somewhat of an analog of branch prediction,
in which paths within the instruction stream, which depend on easily predicted source values, can be
started earlier. As with branch prediction, there must be a way to recover from mispredictions. Another
method that is currently being proposed to reduce the impact of true data dependencies is the use of
simultaneous multithreading in which instructions from multiple threads are interleaved on a single
processor; of course, instructions from different threads are independent by definition.
Out-of-Order Completion
All processors that attempt to execute instructions in parallel must deal with variations in instruction
execution times. That is, some instructions, such as those involving simple integer arithmetic or logic
operations, will need only one cycle for execution, while others, such as floating-point instructions, will
need multiple cycles for execution. If these different instructions are started at the same time, as in a
© 2002 by CRC Press LLC