William Stallings Computer Organization and Architecture 8th EditionMultiple Processor OrganizationSingle Instruction, Single Data Stream - SISDSingle Instruction, Multiple Data Stream - SIMDMultiple Instruction, Single Data Stream - MISDMultiple Instruction, Multiple Data Stream- MIMDTaxonomy of Parallel Processor ArchitecturesMIMD - OverviewTightly Coupled - SMPTightly Coupled - NUMALoosely Coupled - ClustersParallel Organizations - SISDParallel Organizations - SIMDParallel Organizations - MIMD Shared MemoryParallel Organizations - MIMD Distributed MemorySymmetric MultiprocessorsMultiprogramming and MultiprocessingSMP AdvantagesBlock Diagram of Tightly Coupled MultiprocessorOrganization ClassificationTime Shared BusSymmetric Multiprocessor OrganizationTime Share Bus - AdvantagesTime Share Bus - DisadvantageOperating System IssuesA Mainframe SMP IBM zSeriesIBM z990 Multiprocessor StructureCache Coherence and MESI ProtocolSoftware SolutionsHardware SolutionDirectory ProtocolsSnoopy ProtocolsWrite InvalidateWrite UpdateMESI State Transition DiagramIncreasing PerformanceMultithreading and Chip MultiprocessorsDefinitions of Threads and ProcessesImplicit and Explicit MultithreadingApproaches to Explicit MultithreadingScalar Processor ApproachesScalar DiagramsMultiple Instruction Issue Processors (1)Multiple Instruction Issue Diagram (1)Multiple Instruction Issue Processors (2)Multiple Instruction Issue Diagram (2)Parallel, Simultaneous Execution of Multiple ThreadsParallel DiagramExamplesPower5 Instruction Data FlowClustersCluster BenefitsCluster Configurations - Standby Server, No Shared DiskCluster Configurations - Shared DiskOperating Systems Design IssuesParallelizingCluster Computer ArchitectureCluster MiddlewareBlade ServersExample 100-Gbps Ethernet Configuration for Massive Blade Server SiteCluster v. SMPNonuniform Memory Access (NUMA)MotivationCC-NUMA OrganizationCC-NUMA OperationMemory Access SequenceCache CoherenceNUMA Pros & ConsVector ComputationVector Addition ExampleApproachesProcessor DesignsApproaches to Vector ComputationChainingComputer OrganizationsIBM 3090 with Vector FacilityWilliam Stallings Computer Organization and Architecture8th EditionChapter 17Parallel ProcessingMultiple Processor Organization•Single instruction, single data stream - SISD•Single instruction, multiple data stream - SIMD•Multiple instruction, single data stream - MISD•Multiple instruction, multiple data stream- MIMDSingle Instruction, Single Data Stream - SISD•Single processor•Single instruction stream•Data stored in single memory•Uni-processorSingle Instruction, Multiple Data Stream - SIMD•Single machine instruction •Controls simultaneous execution•Number of processing elements•Lockstep basis•Each processing element has associated data memory•Each instruction executed on different set of data by different processors•Vector and array processorsMultiple Instruction, Single Data Stream - MISD•Sequence of data•Transmitted to set of processors•Each processor executes different instruction sequence•Never been implementedMultiple Instruction, Multiple Data Stream- MIMD•Set of processors•Simultaneously execute different instruction sequences•Different sets of data•SMPs, clusters and NUMA systemsTaxonomy of Parallel Processor ArchitecturesMIMD - Overview•General purpose processors•Each can process all instructions necessary•Further classified by method of processor communicationTightly Coupled - SMP•Processors share memory•Communicate via that shared memory•Symmetric Multiprocessor (SMP)—Share single memory or pool—Shared bus to access memory—Memory access time to given area of memory is approximately the same for each processorTightly Coupled - NUMA•Nonuniform memory access•Access times to different regions of memroy may differLoosely Coupled - Clusters•Collection of independent uniprocessors or SMPs•Interconnected to form a cluster•Communication via fixed path or network connectionsParallel Organizations - SISDParallel Organizations - SIMDParallel Organizations - MIMD Shared MemoryParallel Organizations - MIMDDistributed MemorySymmetric Multiprocessors•A stand alone computer with the following characteristics—Two or more similar processors of comparable capacity—Processors share same memory and I/O—Processors are connected by a bus or other internal connection—Memory access time is approximately the same for each processor—All processors share access to I/O–Either through same channels or different channels giving paths to same devices—All processors can perform the same functions (hence symmetric)—System controlled by integrated operating system–providing interaction between processors –Interaction at job, task, file and data element levelsMultiprogramming and MultiprocessingSMP Advantages•Performance—If some work can be done in parallel•Availability—Since all processors can perform the same functions, failure of a single processor does not halt the system•Incremental growth—User can enhance performance by adding additional processors•Scaling—Vendors can offer range of products based on number of processorsBlock Diagram of Tightly Coupled MultiprocessorOrganization Classification•Time shared or common bus•Multiport memory•Central control unitTime Shared Bus•Simplest form•Structure and interface similar to single processor system•Following features provided—Addressing - distinguish modules on bus —Arbitration - any module can be temporary master—Time sharing - if one module has the bus, others must wait and may have to suspend•Now have multiple processors as well as multiple I/O modulesSymmetric Multiprocessor OrganizationTime Share Bus - Advantages•Simplicity•Flexibility•ReliabilityTime Share Bus - Disadvantage•Performance limited by bus cycle time•Each processor should have local cache—Reduce number of bus accesses•Leads to problems with cache coherence—Solved in hardware - see laterOperating System Issues•Simultaneous concurrent processes•Scheduling•Synchronization•Memory management•Reliability and fault toleranceA Mainframe SMPIBM zSeries•Uniprocessor with one main memory card to a high-end system with 48 processors and 8 memory cards•Dual-core processor chip—Each includes two identical central processors (CPs)—CISC superscalar microprocessor—Mostly hardwired, some vertical microcode—256-kB L1 instruction cache and a 256-kB