U. Glaeser

one entrance but may have several exits. This creates a long stream of code, which would normally be
executed sequentially, and allows the compiler to choose instructions over this larger range. Roll-back
code must be added to the hyperblock’s exit to undo the affects of superfluous code that was executed,
but would not execute sequentially. Loops are frequently unrolled, several iterations considered as straight
code, to form hyperblocks. Branch prediction can also help create beneficial hyperblocks.
A newer approach is to dynamically pack the VLIW, using a preprocessor that accesses multiple queues,
one for each functional unit. Realize that using queues is similar to out of order execution.
Interconnection Network
An interconnection network is a necessary component of all parallel processing systems. Several features
govern the choice of a network. A scalable interconnection network for parallel processor would be ideal
if it meets the following requirements for a large range of system sizes. For instance, it may be scalable
by reasonably small increments from 2 4 to perhaps 2 20 processors.
• Have a low average and maximum diameter (the distance between the furthest two nodes) to
avoid communication latency.
• Minimize routing constraints (have many routes from A to B).
• Have a constant number of I/O port (channels) per node to allow for expansion without retrofit.
• Have a simple wire layout to allow for expansion and to avoid wasting VLSI space.
• Be inherently fault tolerant.
• Be sub-dividable for disjoint multiuser applications.
• Be able to handle a large range of algorithms without undo overhead.
The most popular parallel networks—hypercube, quad tree, fat-tree, binary tree, mesh, and torus—fail
in one or more of these items.
Meshes have a major disadvantage: they lack support for long distance connections. Hypercubes have
excellent connectivity by guaranteeing a maximum distance between any two nodes of log N where N is
the number of nodes. Also, many paths exist between any two nodes making it fault tolerant and amenable
to low contention. However, the number of I/O ports per node is log N. As a system scales, each node
would need to be retrofit to add additional ports. In addition, the wire layout is complex, making this
network expensive in space. Tree structures are popular and have a maximum long distance connection of
O(2 log N). Communications on a tree, however, can be complicated by the fact that although many
neighbors are close, many can only communicate through the root. This causes contention at the root. Fattrees
reduce this contention by increasing the bandwidth, as the network approaches the root [Leiserson85].
The extreme ideal network would allow all nodes to connect to all others. This is not practical for
large system. However, one class of networks, the multistage network, uses an internal switching system
and allows constant access time between any two nodes. Several arrangements are available for multistage
networks. All are similar. One such network, the baseline network, is shown in Fig. 5.19. It can be proved
that a N-by-N network can be totally connected with log N switches steps [Seigel89]. Each step is through
a row of switches, with N/2 switches in each row. This requires quite a bit of hardware and does allow
for a connection in log N steps. This is not very scalable.
Optical technology [Yuan97] has shown to be promising for the implementation of all networks.
Instead of a wire, an optical “beam” is used to make the connection. This is fast and has several advantages.
One, a broadcast can be made from one node to many nodes at once (although, one node cannot receive
many inputs at once), and two, the transmission can be sent through clear space.
Conclusion
Parallel systems are going to become almost universal in computer systems. Desktop computers are now
frequently being delivered with more than one CPU and definitely with more than one functional unit.
The new models of SUN, MIPS, Intel, and Macintosh desktop computers currently are providing parallel
© 2002 by CRC Press LLC