Non-linear memory layout transformations and data prefetching ...

2.2 Cache misses 13During a memory reference, a request for specic data can be satised from the cache withoutusing the main memory, when the requested memory address is present in cache. This situationis known as a cache hit. The opposite situation, when the cache is consulted and found not tocontain the desired data, is known as a cache miss. In the latter case, the block of data (wherethe requested element belongs), is usually inserted into the cache, ready for the next access.This block is the smaller amount of elements that can be transferred from one memory level toanother. This block of data is alternatively called cache line.Memory latency and bandwidth are the two key factors that determine the time needed toresolve a cache miss. Memory latency species the intermediate time between the data requestto main memory and the arrival into the cache of the rst data element in the requested block.Memory bandwidth species the arrival rate of the remaining elements in the requested block.A cache miss resolution is critical, because the processor has to stall until the requested dataarrive from main memory (especially in an in-order execution).2.2 Cache missesCache memories were designed to keep the most recently used piece of content (either a programinstruction or data). However, it is not feasible to satisfy all data requests. In a case of a cachemiss in instruction cache, the processor stall is resolved when the requested instruction is fetchedfrom main memory. A cache read miss (data load instruction) can be less severe as there canbe other instructions not dependant on the expected data element. Execution is continued untilthe operation which really needs the loaded data is ready to be executed. However, data is oftenused immediately after the load instruction. The last case is a cache write miss, it is the leastserious miss because there are write buers usually, to store data until they are transferred tomain memory or a block is allocated in the cache. The processor can continue until the bueris full.In order to lower cache miss rate, a great deal of analysis has been done on cache behavior inan attempt to nd the best combination of size, associativity, block size, and so on. Sequences ofmemory references performed by benchmark programs are saved as address traces. Subsequentanalysis simulates many dierent possible cache designs on these long address traces. Makingsense of how the many variables aect the cache hit rate can be quite confusing. One signicantcontribution to this analysis was made by Mark Hill, who separated misses into three categories(known as the Three Cs):There are three dierent types of cache misses.• Compulsory misses: are those misses caused by the very rst reference to a block ofdata. They are alternatively called cold-start or rst-reference misses. Cache capacityand associativity do not aect the number of compulsory misses that come up by an