CS 152 Computer Architecture and Engineering Lecture 14 Cache II 2006 10 17 John Lazzaro A nd a fin a ls o l p ro i nt r o du je ct ctio n www cs berkeley edu lazzaro TAs Udam Saini and Jue Sun www inst eecs berkeley edu cs152 CS 152 L14 Cache II UC Regents Fall 2006 UCB 1 Last Time Locality encourages caching Memory Address one dot per access Bad Temporal Locality Spatial Locality Time CS 152 L14 Cache II Donald J Hatfield Jeanette Gerald Program Restructuring for Virtual Memory IBM Systems Journal 10 3 168 192 1971 UC Regents Fall 2006 UCB 2 Today Caches Reloaded Cache misses and performance how do we size the cache Practical cache design a state machine and a controller Write buffers and caches A nd fin a a ls o l p ro i nt r j o du e ct ctio n The cache DRAM interface CS 152 L14 Cache II UC Regents Fall 2006 UCB 3 Recall Color coding main memory Block Blocks of a certain color may only appear in one line of the cache 32 byte blocks 0 1 2 32 bit Memory Address 31 7 6 Which block 25 bits 5 4 3 0 Color Byte 2 bits 5 bits 4 5 6 7 Cache index 27 2 1 CS 152 L14 Cache II UC Regents Fall 2006 UCB 4 Recall A Direct Mapped Cache 31 7 Cache Tag 25 bits 6 5 Index Ex 0x01 24 Cache Tags 4 0 Byte Select Ex 0x00 Cache Data 0 Byte 31 Byte 1 Byte 0 Byte 31 Byte 1 Byte 0 Hit PowerPC 970 64K directmapped Level 1 I cache CS 152 L14 Cache II Valid Bit Return bytes of hit cache line UC Regents Fall 2006 UCB 5 Recall Set Associative Cache N way set associative N is number of blocks for each color Index Cache Tag 26 bits 2 bits Byte Select 4 bits Ex 0x01 Cache Data Valid Cache Tags Cache Tags Valid Cache Block Cache Data Cache Block Cache Block 16 bytes Hit Left Hit Right Cache Block 16 bytes Return bytes of hit set member Cache block halved to keep cached bits constant CS 152 L14 Cache II PowerPC 970 32K 2 way set associative L1 D cache UC Regents Fall 2006 UCB 6 Cache Misses Performance CS 152 L14 Cache II UC Regents Fall 2006 UCB 7 Recall Performance Equation Seconds Program Instructions Program Cycles Instruction Seconds Cycle Earlier computed from Machine CPI 2 Br an ch e 2 th O M ul tip ly er AL U 1 2 St or 5 Lo ad Assumes a constant memory access time True CPI Ideal CPI Memory Stall Cycles See Section 7 3 COD 3e for details CS 152 L14 Cache II True CPI depends on the Average Memory Access Time AMAT for Inst Data AMAT Hit Time Miss Rate x Miss Penalty Goal Reduce AMAT Beware Improving one term may hurt other terms and increase AMAT UC Regents Fall 2006 UCB 8 One type of cache miss Conflict Miss N blocks of same color in use at once but cache can only hold M N of them Miss Rate Solution Increase M Associativity Miss rate improvement equivalent to doubling cache size fully associative Other Solutions Increase number of cache lines blocks in cache Q Why does this help Add a small victim cache that holds blocks recently removed from the cache Q Why does this help Cache Size KB If hit time increases AMAT may go up AMAT Hit Time Miss Rate x Miss Penalty CS 152 L14 Cache II UC Regents Fall 2006 UCB 9 Other causes of cache misses Capacity Misses Compulsory Misses Cache cannot contain all blocks accessed by the program First access of a block by a program Mostly unavoidable Solution Increase size of the cache Solution Prefetch blocks via hardware software Miss rates relative Miss rates absolute Cache Size KB Cache Size KB Also Coherency Misses other processes update memory CS 152 L14 Cache II UC Regents Fall 2006 UCB 10 Thinking about cache miss types What kind of misses happen in a fully associative cache of infinite size A Compulsory misses Must bring each block into cache In addition what kind of misses happen in a finite sized fully associative cache A Capacity misses Program may use more blocks than can fit in cache In addition what kind of misses happen in a set associative or direct map cache A Conflict misses all questions assume the replacement policy used is considered optimal CS 152 L14 Cache II UC Regents Fall 2006 UCB 11 Practical Cache Design CS 152 L14 Cache II UC Regents Fall 2006 UCB 12 Cache Design Datapath Control Datapath for performance control for correctness Most design errors come from incorrect specification of state machine behavior State Machine To CPU Control Control Control Addr To CPU Din Dout Addr Blocks Tags Din Dout To Lower Level Memory To Lower Level Memory Red text will highlight state machine requirements CS 152 L14 Cache II UC Regents Fall 2006 UCB 13 Recall State Machine Design Rst 1 RYG 100 Change 1 Change 1 RYG 001 Change 1 RYG 010 Cache controller state machines like this but more states and perhaps several connected machines CS 152 L14 Cache II UC Regents Fall 2006 UCB 14 Issue 1 Control for CPU interface For reads your state machine must To Processor Upper Level Memory Small fast 1 sense REQ 2 latch Addr 3 create Wait 4 put Data Out on the bus Blk X From Processor Lower Level Memory Large slow Blk Y From CPU To CPU An example interface there are other possibilities CS 152 L14 Cache II UC Regents Fall 2006 UCB 15 Issue 2 Cache Block Replacement After a cache read miss if there are no empty cache blocks which block should be removed from the cache The Least Recently Used A randomly chosen block LRU block Appealing Easy to implement how but hard to implement well does it work Miss Rate for 2 way Set Associative Cache Size Random LRU 16 KB 5 7 2 0 1 17 5 2 1 9 1 15 64 KB 256 KB Also try other LRU approx Part of your state machine decides which block to replace CS 152 L14 Cache II UC Regents Fall 2006 UCB 16 Issue 3 High performance block fetch 1 12 bit row address input o f 4 0 9 6 d e c o d e r CS 152 L14 Cache II With proper memory layout one row access delivers entire cache block to the sense amp Two state machine challenges 1 Bring in the word requested by CPU with lowest latency 2 Bring in rest of cache block ASAP 2048 columns Each column 4096 33 554 432 usable bits 4 bits rows tester found good bits in bigger array deep 8196 bits delivered by sense amps Select requested bits send off the chip UC Regents Fall 2006 UCB 17 fou r or th e last d esired of a lon ger b u rst Th e 256Mb SDRAM u …
View Full Document
Unlocking...