5_15_25_35_45_5Chapter 5Large and Fast: Exploiting Memory HierarchyChapter 5 — Large and Fast: Exploiting Memory Hierarchy — 2Memory Technology Static RAM (SRAM) 0.5ns – 2.5ns, $2000 – $5000 per GB Dynamic RAM (DRAM) 50ns – 70ns, $20 – $75 per GB Magnetic disk 5ms – 20ms, $0.20 – $2 per GB Ideal memory Access time of SRAM Capacity and cost/GB of disk§5.2 Memory TechnologiesDRAM Technology Data stored as a charge in a capacitor Single transistor used to access the charge Must periodically be refreshed Read contents and write back Performed on a DRAM “row”Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 3Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 4DRAM Generations050100150200250300'80 '83 '85 '89 '92 '96 '98 '00 '04 '07TracTcacYear Capacity $/GB1980 64Kbit $15000001983 256Kbit $5000001985 1Mbit $2000001989 4Mbit $500001992 16Mbit $150001996 64Mbit $100001998 128Mbit $40002000 256Mbit $10002004 512Mbit $2502007 1Gbit $50Chapter 6 — Storage and Other I/O Topics — 5Flash Storage Nonvolatile semiconductor storage 100× – 1000× faster than disk Smaller, lower power, more robust But more $/GB (between disk and DRAM)§6.4 Flash StorageChapter 6 — Storage and Other I/O Topics — 6Disk Storage Nonvolatile, rotating magnetic storage§6.3 Disk StorageChapter 5 — Large and Fast: Exploiting Memory Hierarchy — 7Principle of Locality Programs access a small proportion of their address space at any time Temporal locality Items accessed recently are likely to be accessed again soon e.g., instructions in a loop, induction variables Spatial locality Items near those accessed recently are likely to be accessed soon E.g., sequential instruction access, array data§5.1 IntroductionChapter 5 — Large and Fast: Exploiting Memory Hierarchy — 8Taking Advantage of Locality Memory hierarchy Store everything on disk Copy recently accessed (and nearby) items from disk to smaller DRAM memory Main memory Copy more recently accessed (and nearby) items from DRAM to smaller SRAM memory Cache memory attached to CPUChapter 5 — Large and Fast: Exploiting Memory Hierarchy — 9Memory Hierarchy Levels Block (aka line): unit of copying May be multiple words If accessed data is present in upper level Hit: access satisfied by upper level Hit ratio: hits/accesses If accessed data is absent Miss: block copied from lower level Time taken: miss penalty Miss ratio: misses/accesses= 1 – hit ratio Then accessed data supplied from upper levelChapter 5Large and Fast: Exploiting Memory HierarchyChapter 5 — Large and Fast: Exploiting Memory Hierarchy — 2Cache Memory Cache memory The level of the memory hierarchy closest to the CPU Given accesses X1, …, Xn–1, Xn§5.3 The Basics of Caches How do we know if the data is present? Where do we look?Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 3Direct Mapped Cache Location determined by address Direct mapped: only one choice (Block address) modulo (#Blocks in cache) #Blocks is a power of 2 Use low-order address bitsChapter 5 — Large and Fast: Exploiting Memory Hierarchy — 4Tags and Valid Bits How do we know which particular block is stored in a cache location? Store block address as well as the data Actually, only need the high-order bits Called the tag What if there is no data in a location? Valid bit: 1 = present, 0 = not present Initially 0Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 5Cache Example 8-blocks, 1 word/block, direct mapped Initial stateIndex V Tag Data000 N001 N010 N011 N100 N101 N110 N111 NChapter 5 — Large and Fast: Exploiting Memory Hierarchy — 6Cache ExampleIndex V Tag Data000 N001 N010 N011 N100 N101 N110 Y 10 Mem[10110]111 NWord addr Binary addr Hit/miss Cache block22 10 110 Miss 110Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 7Cache ExampleIndex V Tag Data000 N001 N010 Y 11 Mem[11010]011 N100 N101 N110 Y 10 Mem[10110]111 NWord addr Binary addr Hit/miss Cache block26 11 010 Miss 010Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 8Cache ExampleIndex V Tag Data000 N001 N010 Y 11 Mem[11010]011 N100 N101 N110 Y 10 Mem[10110]111 NWord addr Binary addr Hit/miss Cache block22 10 110 Hit 11026 11 010 Hit 010Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 9Cache ExampleIndex V Tag Data000 Y 10 Mem[10000]001 N010 Y 11 Mem[11010]011 Y 00 Mem[00011]100 N101 N110 Y 10 Mem[10110]111 NWord addr Binary addr Hit/miss Cache block16 10 000 Miss 0003 00 011 Miss 01116 10 000 Hit 000Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 10Cache ExampleIndex V Tag Data000 Y 10 Mem[10000]001 N010 Y 10 Mem[10010]011 Y 00 Mem[00011]100 N101 N110 Y 10 Mem[10110]111 NWord addr Binary addr Hit/miss Cache block18 10 010 Miss 010Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 11Address SubdivisionChapter 5Large and Fast: Exploiting Memory HierarchyChapter 5 — Large and Fast: Exploiting Memory Hierarchy — 2Example: Larger Block Size 64 blocks, 16 bytes/block To what block number does address 1200 map? Block address = 1200/16 = 75 Block number = 75 modulo 64 = 11Tag Index Offset034910314 bits6 bits22 bitsChapter 5 — Large and Fast: Exploiting Memory Hierarchy — 3Block Size Considerations Larger blocks should reduce miss rate Due to spatial locality But in a fixed-sized cache Larger blocks  fewer of them More competition  increased miss rate Larger blocks  pollution Larger miss penalty Can override benefit of reduced miss rate Early restart and critical-word-first can helpChapter 5 — Large and Fast: Exploiting Memory Hierarchy — 4Cache Misses On cache hit, CPU proceeds normally On cache miss Stall the CPU pipeline Fetch block from next level of hierarchy Instruction cache miss Restart instruction fetch Data cache miss Complete data accessChapter 5 — Large and Fast: Exploiting Memory Hierarchy — 5Write-Through On data-write hit, could just update the block in cache But then cache and memory would be inconsistent Write through: also update memory But makes writes take longer e.g., if base CPI = 1, 10% of instructions are stores, write to memory takes 100 cycles Effective CPI = 1 + 0.1×100 = 11 Solution: write buffer