Performance Analysis and Optimization of the Hurricane File System ...

CHAPTER 5. MICROBENCHMARKS – READ OPERATION 36
16 hash lists to be used, resulting in 8 hash lists that were shared. For example, processors 0 to 3 shared
hash lists with processors 8 to 11. The wide variability on 12 processors, as shown by the large error bars
in Figure 5.2, was the result of some threads sharing hash lists while other threads had exclusive access to
hash lists. Nevertheless, the performance degradation on a contended hash list was higher than expected,
considering that only 2 processors were sharing any contended hash list. Performance did not simply slow
down by a factor of 2, as one may intuitively expect. The choice of lock types was examined in a later
experiment to determine if it was the culprit behind this severity. Regardless, it was necessary to modify the
ORS hash function to reduce hash list collisions under a variety of workloads. It was surprising that hash
collisions occurred under the simple microbenchmark workload.
Figure 5.4 magnifies the left portion of Figure 5.2. It shows that there is also a serious performance
degradation between 1 processor to 2 processors. The costs of transitioning from a uniprocessor to multi-
processor architecture can be clearly seen. Overheads may include memory bus contention, memory bank
contention, multiprocessor locks, cache-coherence misses, and cache-coherence traffic. These problems do
not exist on uniprocessors. As a result, two concurrent threads required twice as long to complete their
independent tasks compared to one thread. Performance between 2 processors and 8 processors showed ideal
scalability since hash collisions did not occur in this range of processors.
5.4 Optimizations
We implemented a number of optimizations to improve scalability. These optimizations include (1) a mod-
ified ORS hash function, (2) larger ORS hash table, and (3) padded ORS hash list headers and cache
entries. The fully optimized results are shown in Figure 5.5 1 . This section describes the investigation
process that led to these optimizations. We used the perfect memory model in Section 5.4.1 to examine
memory bottlenecks. A base line experiment was run in Section 5.4.2 to verify scalability of the underlying
infrastructure. The impact of lock types and cache associativity were examined in Sections 5.4.3 and 5.4.4,
respectively. As described in Section 5.4.5, modifying the ORS hash function alone was not adequate and
introduced other performance problems. Section 5.4.6 indicates that padding the ORS hash list headers
alleviated these problems. Increasing the number of hash lists (increasing the size of the hash table) can
be done without detrimental effects, as shown in Section 5.4.7. Further padding of the ORS cache entries
significantly improved absolute performance in Section 5.4.8. In the investigation process, we discovered a
potential problem with the K42 padding and alignment facilities. Finally, Section 5.4.9 describes the fully
optimized configuration in detail.
The experimental configurations used here are based on those used in Section 5.2 with minor modifications
as described in each subsection. We continue to use extent-based files in the experiments to explore ORS
1 Error bars were removed from the unoptimized 12 processor configuration.