CMSC 411CMSC 411Computer Systems ArchitectureLecture 22Multiprocessors, cont.Multiprocessors, cont.Alan Sussmanl@ d [email protected]• Finish reading Chapter 4•Exam 2 answers postedquestions?•Exam 2 answers posted –questions?– Mean: 60 Median: 60 StdDev: 12• Cache simulator project due tomorrowti ?–questions?• Course evaluations open, at http://www.CourseEvalUM.umd.eduCMSC 411 - 22 (some from Patterson, Sussman, others)2Example Snooping Protocol• Snooping coherence protocol is usually implemented by incorporating a finite-state tll i hd( h)controller in each node (cache)• Logically, think of a separate controller associated with each cache block– That is, snooping operations or cache requests for different blocks can proceed independently•Inreal implementations, a single controller allowsIn real implementations, a single controller allows multiple operations to distinct blocks to proceed in interleaved fashion –meaningone operation may be initiated before another ismeaning one operation may be initiated before another is completed, even though only one cache access or one bus access is allowed at a time CMSC 411 - 22 (some from Patterson, Sussman, others)3Write-through Invalidate Protocolg• 2 states per block in each cache– as in uniprocessor–state of a block is a p-vector of states– Hardware state bits associated with blocks that are in the cache other blocks can be seen as being in–other blocks can be seen as being in invalid (not-present) state in that cache• Writes invalidate all other cache copiescopies– can have multiple simultaneous readers of block, but write invalidates themState Tag Data State Tag DataP1cachecachePnBCMSC 411 - 22 (some from Patterson, Sussman, others)4I/O devicesMemBusThe ExampleFormat: event / actionPrRd: Processor ReadPrWr: Processor WriteBusRd: Bus ReadVPrRd/ --PrWr / BusWrBusRd: Bus ReadBusWr: Bus WriteVBusWr / -IPrWr/BusWrPrRd / BusRdPrWr/ BusWrSi lit lti ld(SWMR)CMSC 411 - 22 (some from Patterson, Sussman, others)5Single-writer, multiple-reader (SWMR)Is Example 2-state Protocol Coherent?Memory is coherent iff:1A read by P to location X thatfollowsawritebyPtoX w/1.A read by P to location X that followsa write by P to X, w/ no intervening writes, returns written value.2. A read by P to location X that follows a write by Q to X, w/ no intervening writes, and the read and write sufficiently separated, returns written value.3. Writes to same location are serialized• writes to same location by distinct processors seen in same order by all other processorsCMSC 411 - 22 (some from Patterson, Sussman, others)6Is Example 2-state Protocol Coherent?Assume bus transactions and memory ops atomic, and a one-level cache– all phases of one bus transaction complete before next one startsit f ti t l t b f i i t–processor waits for memory operation to complete before issuing next– with one-level cache, assume invalidations applied during bus transactionProcessors only observe state of memory through reads….Writes only observable by other processors if on bus…•All writes go to bus! (in this example protocol not all others)•All writes go to bus! (in this example protocol, not all others)– Writes serialized by order in which they appear on bus (bus order)– invalidations applied to caches in bus order• How to insert reads in this order?– Important since processors see writes through reads, so determines whether write serialization is satisfiedBut read hits may happen independently and do not appear on bus or enter directly inCMSC 411 - 22 (some from Patterson, Sussman, others)7–But read hits may happen independently and do not appear on bus or enter directly in bus orderOrderingWRRRP0:RRWRRR RWRRRRP0:P1:RRRR RRR RP2:RR• Writes establish a partial order• Doesn’t constrain ordering of reads, though shared-medium (bus) will order read misses too– any order among reads between writes is fineCMSC 411 - 22 (some from Patterson, Sussman, others)8• Writes serialized, reads and writes not interchanged, so coherent!Outline• Review• Coherence• Write Consistency• Administrivia•Snooping•Snooping• Building Blocks• Snooping protocols and examplespgp p• Coherence traffic and Performance on MP• Directory-based protocols and examples Cli•ConclusionCMSC 411 - 22 (some from Patterson, Sussman, others)9Example Write Back Snoopy Protocol• Invalidation protocol, write-back cache– Snoops every address on bus– If it has a dirty copy of requested block, provides that block in response to the read request and aborts the memory access• Each memory block is in one state:– Clean in all caches and up-to-date in memory (Shared)– OR Dirty in exactly one cache (Exclusive)– OR Not in any caches• Each cache block is in one state (track these):– Shared : block can be read–OR Exclusive : cache has only copy, its writeable, and dirtyyy y– OR Invalid : block contains no data (in uniprocessor cache too)• Read misses: cause all caches to snoop busWrites to clean blocks are treated as missesCMSC 411 - 22 (some from Patterson, Sussman, others)10•Writes to clean blocks are treated as misses– write-allocateWrite-Back State Machine – CPU EventsCPU Read hit• State machine for CPUrequests for each cache blockCPU R d• Non-resident blocks invalidInvalidShared(read/only)CPU ReadPlace read misson busCPU WritePlace Write Mi bMiss on busCPU WriteExclusive(read/write)Place Write Miss on BusCPU read hitCMSC 411 - 22 (some from Patterson, Sussman, others)11CPU write hitWrite-Back State Machine- Bus eventsState machine for busrequests for each cache blockInvalidSharedWrite missfor this blockInvalid(read/only)for this blockWrite missfor this blockWrite BackBlock; (abortmemory access)Read miss for this blockWit B kExclusive(read/write)Write BackBlock; (abort anymemory access)CMSC 411 - 22 (some from Patterson, Sussman, others)12Block-replacementSh dCPU ReadCPU Read hitlocal misses to conflictingblocksInvalidShared(read/only)CPU ReadPlace read misson busconflictingblocksCPU WritePlace Write Miss on busCPU read missWrite back block,Place read misson busCPU Read missPlace read missMiss on buson busCPU WritePlace read miss on busChBlkExclusive(read/write)Place Write Miss on BusCPU read hitCache BlockStateCMSC 411 - 22 (some from Patterson, Sussman, others)13CPU Write MissWrite back cache blockPlace write miss on busCPU read hitCPU write hitWrite-back State Machine-III • State