CS 152 Computer Architecture and Engineering Lecture 14 Cache II A An nd d aals f fn o naall lso p prro 2006 10 17 ojjeec iin nttrro ctt od du John Lazzaro o ucctti o n n www cs berkeley edu lazzaro i TAs Udam Saini and Jue Sun www inst eecs berkeley edu cs152 CS 152 L14 Cache II UC Regents Fall 2006 UCB Last Time Locality encourages caching Memory Address one dot per access Ba d Temporal Locality Spatial Locality Time CS 152 L14 Cache II Donald J Hatfield Jeanette Gerald Program Restructuring for Virtual Memory IBM Systems Journal UC Regents Fall 2006 UCB 10 3 168 192 1971 Today Caches Reloaded Cache misses and performance how do we size the cache Practical cache design a state machine and a controller A An nd d aals Write buffers and caches f fn o naall lso p prro ojjeec iin nttrro ctt od du o ucctti o n The cache DRAM interface n i CS 152 L14 Cache II UC Regents Fall 2006 UCB Recall Color coding main memory Block 32 byte blocks Blocks of a certain color may only appear in one line of the cache 32 bit Memory Address 31 7 6 Which block 25 bits 5 4 0 1 2 3 0 Color Byte 2 bits 5 bits 4 5 6 7 Cache index 27 2 1 CS 152 L14 Cache II UC Regents Fall 2006 UCB Recall A Direct Mapped Cache 31 7 Cache Tag 25 bits 24 Cache Tags 6 5 Index Ex 0x01 0 4 0 Byte Select Cache Data Ex 0x00 Byte 31 Byte 1 Byte 0 Byte 31 Byte 1 Byte 0 Hit PowerPC 970 64K direct mapped LevelCS 152 L14 Cache II 1 I cache Valid Bit Return bytes of hit cache line UC Regents Fall 2006 UCB Recall Set Associative Cache N way set associative N is number of blocks for each color Byte Select Index Cache Tag 26 bits 4 bits 2 bits Ex 0x01 Cache Data ValidCache Tags Cache TagsValid Cache Data Cache Block Cache Block Cache Block 16 bytes Hit Left Hit Right Cache Block 16 bytes Return bytes of hit set member Cache block halved to keep cached bits CS 152 L14 Cache II PowerPC 970 32K 2way UC Regents Fall 2006 UCB Cache Misses Performance CS 152 L14 Cache II UC Regents Fall 2006 UCB Recall Performance Equation Seconds Program Instructions Program Earlier computed from Machine CPIa Assumes constant memory access time Cycles Instruction Seconds Cycle True CPI depends on the Average Memory Access Time AMAT for Inst Data AMAT Hit Time Miss Rate x Miss Penalty Goal Reduce AMAT True CPI Ideal CPI See Section 7 3 Memory Stall COD 3e Cycles for CS 152 L14 Cache II Beware Improving one term may hurt other terms and increase AMAT UC Regents Fall 2006 UCB One type of cache miss Conflict Miss N blocks of same color in use at once but cache can only hold M N of them Miss Rate Solution Increase M Associativity Miss rate improveme nt equivalent to doubling cache size fully associative Other Solutions Increase number of cache lines blocks in cache Q Why does this help Add a small victim cache that holds blocks recently removed from the cache Q Why does this help Cache Size KB If hit time increases AMAT may go up AMAT Hit Time Miss Rate x Miss Penalty CS 152 L14 Cache II UC Regents Fall 2006 UCB Other causes of cache misses Capacity Misses Compulsory Misses Cache cannot contain all blocks accessed by the program First access of a block by a program Mostly unavoidable Solution Increase size of the cache Solution Prefetch blocks via hardware software Miss rates absolute Cache Size KB Miss rates relative Cache Size KB Also Coherency Misses other processes update memory CS 152 L14 Cache II UC Regents Fall 2006 UCB Thinking about cache miss types What kind of misses happen in a fully associative cache of infinite size A Compulsory misses Must bring each block into cache In addition what kind of misses happen in a finite sized fully associative cache A Capacity misses Program may use more blocks than can ft in cache In addition what kind of misses happen in a set associative or direct map cache A Conflict misses all questions assume the replacement policy used is considered optimal CS 152 L14 Cache II UC Regents Fall 2006 UCB Practical Cache Design CS 152 L14 Cache II UC Regents Fall 2006 UCB Cache Design Datapath Control Datapath for performance control for correctness Most design errors come from incorrect specifcation of state machine behavior To CPU State Machine Control Control Control Addr To CPU Din Dout Addr Blocks Tags Din Dout Red text will highlight state machine CS 152 L14 Cache II requirements To Lower Level Memory To Lower Level Memory UC Regents Fall 2006 UCB Recall State Machine Design Rst 1 RYG 100 Change 1 Change 1 RYG 001 Change 1 RYG 010 Cache controller state machines like this but more states and perhaps several CS 152 L14 Cache II UC Regents Fall 2006 UCB Issue 1 Control for CPU interface For reads your state machine must Small fast Large slow 1 sense REQ 2 latch Addr 3 create Wait 4 put Data Out on the From bus CPU To CPU An example interface there are other CS 152 L14 Cache II possibilities UC Regents Fall 2006 UCB Issue 2 Cache Block Replacement After a cache read miss if there are no empty cache blocks which block should be removed from the cache The Least Recently A randomly chosen Used LRU block block Appealing Easy to implement but hard to for 2 way Set Associative Cachehow Miss Rate implement well does it work Also try other LRU appro x Part of your state machine decides which block Size Random LRU 16 KB 5 7 5 2 64 KB 2 0 1 9 256 KB 1 17 1 15 CS 152 L14 Cache II to replace UC Regents Fall 2006 UCB Issue 3 High performance block fetch 1 12 bit row address input of 40 96 de co de r CS 152 L14 Cache II With proper memory layout one row access delivers entire cache block to the sense amp Two state machine challenges 1 Bring in the word requested by CPU with lowest latency 2 Bring in rest of cache block ASAP 2048 columns Each 4096 colum 33 554 432 usable bits rows n4 tester found good bits in bigger array bits deep 8196 bits delivered by sense amps Select requested bits send off UC Regents Fall 2006 UCB Issue 3 continued DRAM Burst Reads One request DRAM can be set up to request an N byte region starting at an arbitrary N k within region Many returns State machine challenges 1 setting up correct block read mode 2 delivering correct word direct to CPU 3 putting all CS 152 L14 Cache II UC Regents Fall 2006 UCB Writes and Caches CS 152 L14 Cache II UC Regents Fall 2006 UCB Issue 4 When to write to lower level Write Through Write …
View Full Document
Unlocking...