Week 8 - WednesdayChapter 5 - Large and Fast: Exploiting Memory Hierarchy5_1Memory Technology Large array of different types of memory for computing Static RAM (SRAM) Typically used in caches Most expensive in price ber byte Lowest latency 0.5ns - 2.5ns, $2000-$5000 per GB Dynamic RAM Main memory Higher latency than SRAM but significantly cheaper 50ns - 70ns, $20-$75 per GB Magnetic disk Hard drives on an actual device Less cost but extreme latency 5ms - 20ms, $0.20-$2 per GB Difficult to get ideal memory that have rapid access time Access time of SRAM Capacity and cost/GB of disk There has to be hiearchy to overcome this problemDRAM Technology Implemented as a single capacitor per bit When read data out of capacitor, have to charge back into it upon read Refreshing adds to the latency Volatile - lose power lose data Higher capacity than cache but not as high as hard disk Between cache and hard disk for latencyFlash Storage Nonvolatile, semiconductor storage Doesn't lose data when it's turned off Faster to access than hard disk A little bit more expensive Between disk and DRAM Can't get same capacity like disk Used as conventional main memory, lacks latencyDisk Storage Nonvolatile, rotating magnetic storage All the rotation contributes to the latencyPrinciple of Locality How to build around these limitations? Porgrams access a small proportion of their address space at any time Locality aspect means that accessing of data over and over again Temporal locality - exactly the same data Items accessed recently are likely to be accessed again soon Instructions in a loop Induction variables Spatial locality - things that are close together in time Items near those accessed recently are likely to be accessed soon Sequential instruction access Array dataTaking Advantage of Locality Form a memory hierarchy Highest point is cache memory First place for memory access Small but high turnaround speed (low latency) Next layer is larger caches (L2/L3) A little bit larger capacity but turnaround speed willnot be as fast as L1 Main memory Large penalty for access time Greater capacity means harder to find Disk Everything that we want is here Highest cost in terms of latency If we access something in DRAM or in disk, we'll bring to a higher level in the hierarchy like cache Things that are close to the data Things that are recently accessed We can exploit locality in the address stripMemory Hierarchy Levels Block (aka line): unit of copying May be multiple words If accessed data is present in upper level Hit: access satisfied by upper level Hit ratio: hits/accesses If absentMiss: block copied from lower level Time taken: miss penalty Miss ratio: misses/accesses = 1 - hit ratio Then accessed data supplied from upper level5_2Cache Memory Cache memory The level of the memory hierarchy closest to the CPU Questions: How do we organize the cache memory in the architectural sense to the best extent How do we manage data that goes in and out? Multiple levels of caches L1-L3Direct Mapped Cache Lowest overhead cache to implement Cache can't hold everything 8 entry cache can only hold 8 blocks of instructions Location of where I looked will be determined by the address Only one choice Block address modulo number of blocks in cache Number of blocks is a power of 2 Use low-order address bitsTags and Valid Bits How do we know which particular block is stored in a cache location? Store block address as well as the data We only need the high-order bits This is called tag What if there was no data in a location? This is called valid 1 - present 0 - not present Initially 01Cache Example Table example: Index V Tag Data 000 N 000 N000 N 000 N 000 N 000 N 000 N 000 N Instructions: Word addr Binary addr Hit/Miss Cache block Instruction: 22 10 110 Miss 110 First time access will be a miss Initially not in the cache So we bring it to the cache Upper 2 bits are used for the tag Lower 3 bits are used for the index in the cache Data will be Mem[10110] Table will look like: Index V Tag Data 000 N 001 N 010 N 011 N 100 N 101 N 110 Y 10 Mem[10110] 111 N Instruction: 26 11 010 Miss 010 First time access will be a miss 11 will be tag, 010 will be Index Data will contain contents of Mem[11010] Table will look like: Index V Tag Data 000 N 001 N 010 Y 11 Mem[11010] 011 N 100 N 101 N 110 Y 10 Mem[10110] 111 N Instructions:22 10 110 Hit 110 26 11 010 Hit 010 Second time access will be a hit Won't go into memory Instruction 22: Check index 110 Check tag 10 If the same then it's a hit! Temporal locality Instructions: 16 10 000 Miss 000 3 00 011 Miss 011 16 10 000 Hit 000 Table will look like: Index V Tag Data 000 Y 10 Mem[10000] 001 N 010 Y 11 Mem[11010] 011 Y 00 Mem[00011] 100 N 101 N 110 Y 10 Mem[10110] 111 N First time access will be a miss Second time access will be a hit Instructions: 18 10 010 Miss 010 Table will look like: Index V Tag Data 000 Y 10 Mem[10000] 001 N 010 Y 10 Mem[10010] 011 Y 00 Mem[00011] 100 N 101 N 110 Y 10 Mem[10110] 111 N We have to kick out the previous