U. Glaeser

In contrast to prefetching that masks disk latencies by overlapping the access with that of I/Os to other
disks, caching attempts to exploit temporal locality in the accesses. A selected subset of the recently
accessed blocks are held in the I/O buffer in the expectation that they will be referenced again soon,
thereby avoiding repeated disk accesses for the same block. Although both prefetching and caching are
well-known techniques employed ubiquitously in computer systems and networking, deploying these
mechanisms effectively in a parallel I/O system raises a unique set of challenges.
The I/O schedule determines the set of blocks that are fetched in each parallel I/O operation. The
schedule is constructed dynamically so as to minimize the total number of parallel I/Os. This requires
the scheduler to decide which blocks to prefetch, and, when the need for replacement arises, to decide
which blocks in the buffer to cache and which to evict. Prefetching and caching in parallel I/O systems
is fundamentally different from that in systems with a single disk, and requires the use of substantially
different algorithms [11,32–36]. In a single-disk system, prefetching is used to overlap I/O operations
with CPU computations. This is usually done using asynchronous I/O whereby a computation continues
after making the I/O request without blocking. A stall model for analyzing the performance of overlapped
I/O and computation was proposed in [17] for a single disk system; prefetching and caching algorithms
to minimize stall time as a function of CPU and I/O speeds were presented in [5,17]. Disk scheduling
algorithms that reorder I/O requests to minimize the disk seek times [59] can also be considered as a
form of prefetching in single-disk systems.
In parallel I/O systems prefetching allows overlap between accesses on different disks thereby hiding
the I/O latency behind the access latency on some other disk. The scheduler has to judiciously decide on
questions like how much buffer to allocate for prefetching and how much for caching, which blocks to
prefetch, and which blocks to cache. For instance, to utilize the available bandwidth, it may appear
desirable to keep a large number of disks busy prefetching data during an I/O; however, excessive
prefetching can fill up the buffer with blocks, which may not be used until much later in the computation.
Such blocks have the adverse effects of choking the buffer and reducing the parallelism in fetching more
immediate blocks. In fact, even when the problem does not involve the use of caching, the decisions of
which blocks to prefetch and when to do so is not trivial.
Another issue needs to be addressed to employ prefetching and caching effectively. In order to prefetch
accurately (rather than speculatively) some knowledge of future accesses is required. This is embodied
in the notion of lookahead, which is a measure of the extent of knowledge about the future accesses that
is available in making prefetching and caching decisions. Obtaining this lookahead has been the area of
much active research [13,40,43,50]. In some applications like external sorting the lookahead can be obtained
dynamically by using a sample of the data to accurately predict the sequence of block requests [10]. In
video retrieval the sequence is determined by the playback times of blocks in the set of concurrently
accessed streams; summary statistics of the streams are used to obtain the lookahead at run time [22].
Indexes in database systems can similarly be used to provide information about the actual sequence of
data blocks that must be accessed. In broadcast servers the set of requests are prioritized by the system
to maximize utilization of the broadcast channel [4]; the prioritized request sequence provides the
lookahead for required I/O accesses. Access patterns can be revealed to the system either using programmer
provided hints [50], or the system may attempt to uncover sequential or strided access patterns
automatically at run time [40]. Speculative execution is another technique based on executing program
code speculatively to determine the control path and the blocks accessed in the path [13].
33.5 Limitations of Simple Prefetching
and Caching Strategies
In [11,32], the problem of scheduling read-once reference strings, in which each block is accessed exactly
once, was considered. Such reference strings are characteristic of streaming applications like multimedia
retrieval. Simple intuitive algorithms that work well in a single-disk scenario were analyzed and shown
to have poor performance in the multiple-disk case. For instance, consider a natural scheduling algorithm
© 2002 by CRC Press LLC