Intel® 64 and IA-32 Architectures Optimization Reference Manual

INTEL® 64 AND IA-32 PROCESSOR ARCHITECTURES• Prefetching far ahead can cause eviction of cached data from the caches prior tothe data being used in execution.• Not prefetching far enough can reduce the ability to overlap memory andexecution latencies.Software prefetches are treated by the processor as a hint to initiate a request tofetch data from the memory system, and consume resources in the processor andthe use of too many prefetches can limit their effectiveness. Examples of this includeprefetching data in a loop for a reference outside the loop and prefetching in a basicblock that is frequently executed, but which seldom precedes the reference for whichthe prefetch is targeted.See: Chapter 9, “Optimizing Cache Usage.”Automatic hardware prefetch is a feature in the Pentium 4 processor. It bringscache lines into the unified second-level cache based on prior reference patterns.Software prefetching has the following characteristics:• handles irregular access patterns, which do not trigger the hardware prefetcher• handles prefetching of short arrays and avoids hardware prefetching start-updelay before initiating the fetches• must be added to new code; so it does not benefit existing applicationsHardware prefetching for Pentium 4 processor has the following characteristics:• works with existing applications• does not require extensive study of prefetch instructions• requires regular access patterns• avoids instruction and issue port bandwidth overhead• has a start-up penalty before the hardware prefetcher triggers and beginsinitiating fetchesThe hardware prefetcher can handle multiple streams in either the forward or backwarddirections. The start-up delay and fetch-ahead has a larger effect for shortarrays when hardware prefetching generates a request for data beyond the end of anarray (not actually utilized). The hardware penalty diminishes if it is amortized overlonger arrays.Hardware prefetching is triggered after two successive cache misses in the last levelcache and requires these cache misses to satisfy a condition that the linear addressdistance between these cache misses is within a threshold value. The threshold valuedepends on the processor implementation (see Table 2-6). However, hardwareprefetching will not cross 4-KByte page boundaries. As a result, hardwareprefetching can be very effective when dealing with cache miss patterns that havesmall strides and that are significantly less than half the threshold distance to triggerhardware prefetching. On the other hand, hardware prefetching will not benefitcache miss patterns that have frequent DTLB misses or have access strides thatcause successive cache misses that are spatially apart by more than the triggerthreshold distance.2-30