CMSC 411CMSC 411Computer Systems ArchitectureLecture 12Memory HierarchyMemory HierarchyAlan Sussmanl@ d [email protected]• Homework #3 due Thursday•Read Appendix C of H&P•Read Appendix C of H&P• Wanli’s office hours have changed– new one on class home pageCMSC 411 - 12 (some from Patterson, Sussman, others)2Levels of the Memory HierarchyCapacityA TiUpper LevelCPU Registers100s Bytes<10s nsAccess TimeCostRegistersStagingXfer Unitfaster<10s nsCacheK Bytes10-100 ns10 1 t /bitCacheInstr. Operandsprog./compiler1-8 bytes1-0.1 cents/bitMain MemoryM Bytes200ns500nsMemoryBlockscache cntl8-128 bytes200ns-500ns$.0001-.00001 cents /bitDiskG Bytes, 10 ms (10,000,000 ns)DiskPagesOS512-4K bytes10 - 10 cents/bit-5-6TapeinfiniteTFilesuser/operatorMbytesL LlLargerCMSC 411 - 12 (some from Patterson, Sussman, others)3sec-min10-8TapeLower LevelThe Principle of Locality• The Principle of Locality:–Program accesses a relatively small portion of the address space at gyp pany short period of time.• Two Different Types of Locality:–Temporal Locality(Locality in Time): If an item is referenced, it willTemporal Locality (Locality in Time): If an item is referenced, it will tend to be referenced again soon (e.g., loops, reuse)– Spatial Locality (Locality in Space): If an item is referenced, items whose addresses are close by tend to be referenced soon (e gstraightlinecode array access)(e.g., straightlinecode, array access)• Last 15-20 years, HW has relied on locality for to improve overall performanceIt is a property of programs that is exploited in machine design.CMSC 411 - 12 (some from Patterson, Sussman, others)4Programs with locality cache well ...Bdl lit b h iaccess)Bad locality behaviorTemporale dot per aLocalityress (onemory AddrSpatialLocalityMemDonald J. Hatfield, Jeanette Gerald: Program Restructuring for Virtual Memory. IBM Systems Journal 10(3): 168-192 (1971)Time5CMSC 411 - 12 (some from Patterson, Sussman, others)Issues to consider• How big should the fastest memory (cache memory) be?• How do we decide what to put in cache memory?• If the cache is full, how do we decide what to remove?remove?• How do we find something in cache?• How do we handle writes?odoeade tesCMSC 411 - 12 (some from Patterson, Sussman, others)6First, there is main memory• Jargon:– frame address – which page?– block number – which cache block?– contents – the dataCMSC 411 - 12 (some from Patterson, Sussman, others)7Then add a cache• Jargon: Each address of a memory location is partitioned intopartitioned into– block address» tag»index»index– block offsetFig 5 5CMSC 411 - 12 (some from Patterson, Sussman, others)8Fig. 5.5How does cache memory work?• The following slides discuss:– what cache memory is– three organizations for cache memory» direct mapped.» set associative» fully associative– how the bookkeeping is done• Important note: All addresses shown are in po ta t ote add esses s o a eoctal. Addresses in the book are usually decimal.CMSC 411 - 12 (some from Patterson, Sussman, others)9What is cache memory? Main memory firstMain memory is divided into (cache) blocks.Each block contains many words (32-256 common now).CMSC 411 - 12 (some from Patterson, Sussman, others)10Main memoryBlocks are grouped into frames (pages), 3 frames in this picture.CMSC 411 - 12 (some from Patterson, Sussman, others)11Main memory (cont.)Blocks are addressed by their frame number and their block number0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 2 2 2Blocks are addressed by their frame number, and their block numberwithin the frame.0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2CMSC 411 - 12 (some from Patterson, Sussman, others)12Cache memoryCache has many, MANY fewer bl k th i h ithblocks than main memory, each witha block number,0 1 2 3 4 5 6 7a memory address,10 21 42 53 74 25 16 77data,a valid bit,0 0 0 0 0 0 0 000000000CMSC 411 - 12 (some from Patterson, Sussman, others)13a dirty bit.00000000Cache memory (cont.)i i ll ll hbiInitially, all the validbitsset to zero.0 1 2 3 4 5 6 710 21 42 53 74 25 16 770 0 0 0 0 0 0 000000000CMSC 411 - 12 (some from Patterson, Sussman, others)1400000000Where can a block be placed?• Block 12 placed in 8 block cache:Fll iti di t d2titi–Fully associative, direct mapped, 2-way set associativeDi t M d2W Ass01234567 0123456701234567Full MappedDirect Mapped(12 mod 8) = 42-Way Assoc(12 mod 4) = 0CacheMemory111111111122222222223301234567890123456789012345678901CMSC 411 - 12 (some from Patterson, Sussman, others)15MemoryCache memory (cont.)0 1 2 3 4 5 6 71021425374251677Suppose want to load block 14 (octal) from memory into cache1021425374251677cache.Three ways to organize hcache• direct mapped• set associative0 0 0 0 0 0 0 00 0 0 0 0 0 0 0• fully associativeCMSC 411 - 12 (some from Patterson, Sussman, others)16Direct mapped cache0 1 2 3 4 5 6 71021425374251677In direct mapped cache, block 14 can only be put in the cache block with1021425374251677the cache block with address 4.So the cache will no longer hold the block with memory address 74.0 0 0 0 0 0 0 00 0 0 0 0 0 0 0yCMSC 411 - 12 (some from Patterson, Sussman, others)17Direct mapped cache (cont.)0 1 2 3 4 5 6 710 21 42 53 14 25 16 77After the load, the contents look like this.0 0 0 0 1 0 0 00 0 0 0 0 0 0 0CMSC 411 - 12 (some from Patterson, Sussman, others)18Set associative cacheSet 0 Set 1 Set 2 Set 30 1 2 3 4 5 6 7In set associative cache, each memory block can be put in any of a set of possible blocksSet 0 Set 1 Set 2 Set 310 24 41 55 72 26 13 77any of a set of possible blocks in cache.For example, if divide cache into 4 sets, block 14 can be put in any block in Set 0 0 0 0 0 0 0 0 000000000py(since last two bits of 14 octal are zero).CMSC 411 - 12 (some from Patterson, Sussman, others)1900000000Set associative cache (cont.)()Set 0 Set 1 Set 2 Set 30 1 2 3 4 5 6 71424415572261377So after loading the block, cache memory might looklike this1424415572261377like this.1 0 0 0 0 0 0 000000000CMSC 411 - 12 (some from Patterson, Sussman, others)2000000000Set associative cache (cont.)Note that the last two bits of theSet 0 Set 1 Set 2 Set 30 1 2 3 4 5 6 71424415572261377Note that the last two bits of the memory block’s address always match the set number, so do not need to be stored. This part of the 1424415572261377address is called the index. The higher order bits are stored, and are called the tag.In these pictures both index and tagIn these pictures, both index and