DOC PREVIEW
Berkeley COMPSCI C267 - Lecture 4: Shared Memory Multiprocessors

This preview shows page 1-2 out of 6 pages.

Save
View full document
View full document
Premium Document
Do you want full access? Go Premium and unlock all 6 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 6 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 6 pages.
Access to all documents
Download any document
Ad free experience

Unformatted text preview:

9/10/2001CS267, Yelick 19/10/2001 CS267, YelickCS 267: Applications of Parallel ComputersLecture 4:Shared Memory MultiprocessorsKathy Yelickhttp://www-inst.eecs.berkeley.edu/~cs2679/10/2001 CS267, YelickBasic Shared Memory ArchitectureP1 P2 Pnnetwork$ $ $memory• Processors all connected to a large shared memory• Local caches for each processor• Cost: much cheaper to cache than main memory°°Simple to program, but hard to scaleSimple to program, but hard to scale°°Now take a closer look at structure, costs, limitsNow take a closer look at structure, costs, limits9/10/2001 CS267, YelickProgramming Shared Memory (review)• Program is a collection of threads of control.• Each thread has a set of private variables• e.g. local variables on the stack. • Collectively with a set of shared variables• e.g., static variables, shared common blocks, global heap.• Communication and synchronization through shared variablesiressPPPiress. . .x = ...y = ..x ...Address:SharedPrivate9/10/2001 CS267, YelickOutline• Historical perspective• Bus-based machines• Pentium SMP• IBM SP node• Directory-based (CC-NUMA) machine• Origin 2000• Global address space machines• Cray t3d and (sort of) t3e60s Mainframe Multiprocessors• Enhance memory capacity or I/O capabilities by adding memory modules or I/O devices• How do you enhance processing capacity?• Add processors• Already need an interconnect between slow memory banks and processor + I/O channels• cross-bar or multistage interconnection networkProcI/ODevicesInterconnectProcMemIOCMem Mem MemIOCPIO IOPMMMM70s Breakthrough: Caches• Memory system scaled by adding memory modules• Both bandwidth and capacity• Memory was still a bottleneck• Enter… Caches!• Cache does two things:• Reduces average access time (latency)• Reduces bandwidth requirements to memoryPmemory (slow)interconnectI/O DeviceorProcessorA: 17processor (fast)9/10/2001CS267, Yelick 2Technology PerspectiveDRAMYear Size Cycle Time1980 64 Kb 250 ns1983 256 Kb 220 ns1986 1 Mb 190 ns1989 4 Mb 165 ns1992 16 Mb 145 ns1995 64 Mb 120 ns1000:1! 2:1!Capacity SpeedLogic: 2x in 3 years 2x in 3 yearsDRAM: 4x in 3 years 1.4x in 10 yearsDisk: 2x in 3 years 1.4x in 10 years0501001502002503003501986 1988 1990 1992 1994 1996Ye a rSpecIntSpecFP9/10/2001 CS267, YelickApproaches to Building Parallel MachinesP1SwitchMain memoryPn(Interleaved)(Interleaved)First-level $P1$Interconnection network$PnMemMemP1$Interconnection network$PnMemMemShared CacheCentralized MemoryDance Hall, UMADistributed Memory (NUMA)Scale9/10/2001 CS267, Yelick80s Shared Memory: Shared Cachei80286i80486Pentiumi80386i8086i4004R10000R4400R3010SU MIPS1000100001000001000000100000001000000001965 1970 1975 1980 1985 1990 1995 2000 2005YearTransistorsi80x86M68 KMIP S• Alliant FX-8• early 80’s• eight 68020s with x-bar to 512 KB interleaved cache• Encore & Sequent• first 32-bit micros (N32032)• two to a board with a shared cacheP1SwitchMain memoryPn(Interleaved)(Interleaved)First-level $9/10/2001 CS267, YelickShared Cache: Advantages and DisadvantagesAdvantages• Cache placement identical to single cache• only one copy of any cached block• Fine-grain sharing is possible• Interference• One processor may prefetch data for another• Can share data within a line without moving lineDisadvantages• Bandwidth limitation• Interference• One processor may flush another processors dataLimits of Shared Cache ApproachI/O MEM MEM° ° °PROCcachePROCcache° ° °Assume:1 GHz processor w/o cache=> 4 GB/s inst BW per processor (32-bit)=> 1.2 GB/s data BW at 30% load-storeNeed 5.2 GB/s of bus bandwidth per processor!• Typical bus bandwidth is closer to 1 GB/s5.2 GB/s140 MB/s9/10/2001 CS267, YelickApproaches to Building Parallel MachinesP1SwitchMain memoryPn(Interleaved)(Interleaved)First-level $P1$Interconnection network$PnMemMemP1$Interconnection network$PnMemMemShared CacheCentralized MemoryDance Hall, UMADistributed Memory (NUMA)Scale9/10/2001CS267, Yelick 39/10/2001 CS267, YelickIntuitive Memory Model• Reading an address should return the last value writtento that address• Easy in uniprocessors• except for I/O• Cache coherence problem in MPs is more pervasive and more performance critical• More formally, this is called sequential consistency:“A multiprocessor is sequentially consistent if the result of anyexecution is the same as if the operations of all the processorswere executed in some sequential order, and the operations of each individual processor appear in this sequence in the order specified by its program.” [Lamport, 1979]9/10/2001 CS267, YelickCache Coherence: Semantic Problem• p1 and p2 both have cached copies of x (as 0)• p1 writes x=1 • May “write through” to memory• p2 reads x, but gets the “stale” cached copyx 0x 0x = 0p1 p2x 19/10/2001 CS267, YelickCache Coherence: Semantic ProblemWhat does this imply about program behavior?• No process ever sees “garbage” values, I.e., ½ of 2 values• Processors always see values written by some some processor• The value seen is constrained by program order on all processors• Time always move forward•Example:• P1 writes x=1, then writes y=1• P2 read y, then reads xx = 0y = 0x = 1y = 1= y= xIf P2 sees the new value of y, it must see the new value of xP1P29/10/2001 CS267, YelickSnoopy Cache-Coherence Protocols• Bus is a broadcast medium & caches know what they have• Cache Controller “snoops” all transactions on the shared bus• A transaction is a relevant transaction if it involves a cache block currently contained in this cache• take action to ensure coherence• invalidate, update, or supply value• depends on state of the block and the protocolStateAddressDataI/O devicesMemP1$Bus snoop$PnCache-memorytransaction9/10/2001 CS267, YelickBasic Choices in Cache Coherence• Cache may keep information such as:• Valid/invalid• Dirty (inconsistent with memory)• Shared (in another caches)• When a processor executes a write operation to shared data, basic design choices are:• Write thru: do the write in memory as well as cache• Write back: wait and do the write later, when the item is flushed• Update: give all other processors the new value• Invalidate: all other processors remove from cache9/10/2001 CS267, YelickExample: Write-thru Invalidate• Update and write-thru both use more memory bandwidth if there are writes to the same address• Update to the


View Full Document

Berkeley COMPSCI C267 - Lecture 4: Shared Memory Multiprocessors

Documents in this Course
Lecture 4

Lecture 4

52 pages

Split-C

Split-C

5 pages

Lecture 5

Lecture 5

40 pages

Load more
Download Lecture 4: Shared Memory Multiprocessors
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Lecture 4: Shared Memory Multiprocessors and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Lecture 4: Shared Memory Multiprocessors 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?