Non-linear memory layout transformations and data prefetching ...

More documents

Recommendations

Info

16 Basic Conceptscache entries, it has the disadvantage of slowing down cache access rates due to added complexity.Therefore minimizing the degree of associativity is an important concern. One option is to havea set-associative primary cache, where addresses are mapped to N-way associative sets. Anotherapproach is to keep the rst level cache direct-mapped but also add a victim cache o to the side.The goal is to make the victim cache transparent to system so the rst level cache thinks it isrequesting directly to the DRAM. Victim caching places a small fully-associative cache betweenthe rst level cache and its rell path. This small fully-associative cache is loaded with thevictims of a miss in the L1 cache. In the case of a miss in L1 cache that hits in the victim cache,the contents of the L1 cache line and the matching victim cache line are swapped. Misses inthe cache that hit in the victim cache have only a one cycle miss penalty, as opposed to a manycycle miss penalty without the victim cache. The victim cache approach is appealing becauseit is tailored to cases where data are reused shortly after being displaced-this is precisely whathappens with prefetching conicts.According to measurement [Jou90], a 4-entry victim cache removes 20%-95% of conictmisses for a 4 KByte direct mapped data cache. Typically a small fully associated victim cacheis used with a direct mapped main (rst level) cache. The victim cache usually uses a FIFO(rst in, rst out) replacement policy. The reason for using FIFO instead of random is thatrandom replacement is more eective for larger cache sizes. The small size of the victim cachemakes it probable that certain cache blocks will be replaced more often. A FIFO replacementpolicy ensures each block has the same chance to be replaced. Theoretically a LRU is the bestreplacement policy, but is too dicult to implement with more than two sets.2.3.3 PrefetchingVictim caches and pseudo-associativity both promise to improve miss rates without aectingthe processor clock rate. Prefetching is another technique that predicts soon-to-be used instructionsor data and loads them into the cache before they are accessed. Subsequently, when theprefetched instructions or data are accessed there is a cache hit, rather than a miss.A commonly used method is sequential prefetching, according to which it is predicted thatdata or instructions immediately following those currently accessed, will be needed in the nearfuture and should be prefetched. Sequential prefetching fetches a memory block that caused acache miss, along with the next n consecutive cache blocks. It can be more or less \aggressive"depending on how far ahead in the access stream it attempts to run - that is, how large n is.Both instructions and data can be prefetched, either directly into the caches or into anexternal buer that can be more quickly accessed than main memory. An example of this is theIntel Xeon Pentium 4 processor, which can perform both software and hardware prefetching.The hardware prefetcher requires a couple of misses before it starts operating. It will attempt toprefetch two cache lines ahead of the prefetch stream into the unied second-level cache basedon prior reference patterns. In a cache miss, the hardware mechanism fetches the adjacent
2.4 Cache replacement policies 17cache line within an 128-byte sector that contains the data needed. There is also a softwarecontrolled mechanism that fetches data into the caches using prefetch instructions. The hardwareinstruction fetcher reads instructions that are likely to be executed along the path predicted bythe branch target buer (BTB) into instruction streaming buers.Although usually benecial, prefetching, especially aggressive prefetching may reduce performance.In some cases prefetched data or instructions may not actually be used, but will stillconsume system resources - memory bandwidth and cache space. It is possible that a prefetchwill result in useless instructions or data replacing other instructions or data in the cache thatwill soon be needed. This eect is referred to as cache pollution. The eects of cache pollutionmost often increase as prefetching becomes more aggressive. Another prevalent eect ofaggressive prefetching is bus contention. Bus contention occurs when multiple memory accesseshave to compete to transfer data on the system bus. This eect can create a scenario where ademand-fetch is forced to wait for the completion of a useless prefetch, further increasing thenumber of cycles the processor is kept idle.On average, hardware prefetching benets performance and works with existing applications,without requiring extensive study of prefetch instructions. However it may not be optimal forany particular program especially for irregular access patterns and has a start-up penalty beforethe hardware prefetcher triggers and begins initiating fetches. This start-up delay has a largereect for short arrays when hardware prefetching generates a request for data beyond the endof an array (not actually utilized). There is a software alternative to hardware prefetching,the compiler-controlled prefetching - the compiler requests the data before it is needed. Thistechnique can improve memory access time signicantly, even irregular access patterns, if thecompiler is well implemented and can avoid references that are likely to be cache misses. Howeverit requires new instructions to be added which results to issuing port bandwidth overhead.2.4 Cache replacement policiesWhen a cache miss occurs, the desired data element should be retrieved from memory andplaced in a cache location, according to the cache mapping organization. This may result in theeviction of another blocked data stored in the same location. In direct mapping, there existsonly a single candidate. In set associative and fully associative mappings more than one cachelines are candidate for eviction. The cache line selection is critical, as it aects performance.Which block should be replaced is determined by a replacement policy:The primary strategies used in practice are:• Random: Candidate cache lines are randomly selected. It counts on the principle ofuniform propagation of cache line allocation due to fortuity of selection. This strategy issimple to implement in hardware, but passes over the locality of references.
Page 1: NATIONAL TECHNICAL UNIVERSITY OF AT
Page 4 and 5: .................Evangelia G. Athan
Page 6 and 7: vióôéò ìç-ãñáììéêÝò
Page 8 and 9: viiiAnother issue, that had not bee
Page 10 and 11: xCONTENTS2.7 Iteration Space . . .
Page 12 and 13: xiiCONTENTS
Page 14 and 15: xivLIST OF FIGURES4.4 Alignment of
Page 16 and 17: xviLIST OF FIGURES
Page 18 and 19: xviiiLIST OF TABLES
Page 20 and 21: äýíáìç ðñïò ôç óõíå
Page 22 and 23: 2 Introductionthe processor die its
Page 24 and 25: 4 Introductionthe instruction strea
Page 26 and 27: 6 IntroductionUnfortunately, the pe
Page 28 and 29: 8 Introduction• A study of the ee
Page 30 and 31: 10 Introduction
Page 32 and 33: 12 Basic ConceptsPart of the On-chi
Page 34 and 35: 14 Basic Conceptsapplication execut
Page 38 and 39: 18 Basic Concepts• Least-Recently
Page 40 and 41: 20 Basic Conceptstable. Of course,
Page 42 and 43: 22 Basic ConceptsIn the above examp
Page 44 and 45: 24 Basic Concepts• Forward expres
Page 46 and 47: 26 Basic ConceptsThe following unim
Page 48 and 49: 28 Basic Conceptsfor (i = 0; i
Page 50 and 51: 30 Basic Concepts
Page 52 and 53: 32 Fast Indexing for Blocked Array
Page 76 and 77: 56 A Tile Size Selection Analysis[M
Page 78 and 79: 58 A Tile Size Selection Analysismi
Page 80 and 81: 60 A Tile Size Selection Analysismi
Page 82 and 83: 62 A Tile Size Selection Analysisd.
Page 84 and 85: 64 A Tile Size Selection AnalysisRe
Page 86 and 87:
66 A Tile Size Selection AnalysisTh
Page 88 and 89:
68 A Tile Size Selection AnalysisSu
Page 90 and 91:
70 A Tile Size Selection AnalysisWe
Page 92 and 93:
72 A Tile Size Selection AnalysisTL
Page 94 and 95:
74 A Tile Size Selection Analysis4.
Page 96 and 97:
76 Simultaneous MultithreadingAlong
Page 98 and 99:
78 Simultaneous Multithreadingas be
Page 100 and 101:
80 Simultaneous MultithreadingFor c
Page 102 and 103:
82 Simultaneous Multithreading871th
Page 104 and 105:
84 Simultaneous MultithreadingExami
Page 106 and 107:
86 Experimental Results3500MBalttil
Page 108 and 109:
88 Experimental Results2000MBaLttil
Page 110 and 111:
90 Experimental Results140MBaLtMBaL
Page 112 and 113:
92 Experimental Resultsnumber of mi
Page 114 and 115:
94 Experimental Resultsnumber of mi
Page 116 and 117:
96 Experimental ResultsTotal penalt
Page 118 and 119:
98 Experimental Resultsso that dier
Page 120 and 121:
100 Experimental Results10STRMMSSYR
Page 122 and 123:
102 Experimental ResultsNorm. Perfo
Page 124 and 125:
104 Experimental Resultstool. Figur
Page 126 and 127:
106 Experimental Results
Page 128 and 129:
108 Conclusionsdata are now stored
Page 131 and 132:
APPENDIXATable of SymbolsExplanatio
Page 133 and 134:
APPENDIXBHardware ArchitectureUltra
Page 135 and 136:
APPENDIXCProgram CodesIn the follow
Page 137 and 138:
C.4 SSYMM: Symmetric Matrix-Matrix
Page 139 and 140:
Bibliography[AAKK05]Evangelia Athan
Page 141 and 142:
BIBLIOGRAPHY 121ference on Programm
Page 143 and 144:
BIBLIOGRAPHY 123[KPCM99][KRC97][KRC
Page 145 and 146:
BIBLIOGRAPHY 125[RS01][RT98a][RT98b
Page 147:
BIBLIOGRAPHY 127[WM95][WMC96]Wm. A.
show all

Non-linear memory layout transformations and data prefetching ...

Create successful ePaper yourself

Delete template?

Save as template?