DOC PREVIEW
UMD CMSC 411 - Multiprocessors

This preview shows page 1-2-3 out of 10 pages.

Save
View full document
View full document
Premium Document
Do you want full access? Go Premium and unlock all 10 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 10 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 10 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 10 pages.
Access to all documents
Download any document
Ad free experience

Unformatted text preview:

1Page1Multiprocessors(orParallelComputers)(orParallelProcessors)ParallelComputers• Definition:“Aparallelcomputerisacollectionofprocessingelementsthatcooperateandcommunicatetosolvelargeproblemsfast.”Almasi andGottlieb,HighlyParallelComputing,1989• Questionsaboutparallelcomputers:– Howlargeacollection?– Howpowerfulareprocessingelements?– Howdotheycooperateandcommunicate?– Howaredatatransmitted?– Whattypeofinterconnection?– WhatareHWandSWprimitivesforprogrammer?– Doesittranslateintoperformance?ParallelProcessors“Religion”• Thedreamofcomputerarchitectssince1960:replicateprocessorstoaddperformancevs.designafasterprocessor• Ledtoinnovativeorganizationtiedtoparticularprogrammingmodelssince“uniprocessors can’tkeepgoing”– e.g.,uniprocessors muststopgettingfasterduetolimitofspeedoflight:1972,… ,1989– Bordersreligiousfervor:youmustbelieve!– Fervordampedsomewhen1990scompanieswentoutofbusiness:ThinkingMachines,KendallSquare,...• Argumentinsteadisthe“pull” ofopportunityofscalableperformance,notthe“push” ofuniprocessor performanceplateauOpportunities:ScientificComputing• NearlyUnlimitedDemand(GrandChallenge):App Perf (GFLOPS) Memory(GB)48hourweather 0.1 0.172hourweather 3 1Pharmaceuticaldesign 100 10GlobalChange,Genome 1000 1000Successesinsomerealindustries:– Petroleum:reservoirmodeling– Automotive:crashsimulation,draganalysis,engine– Aeronautics:airflowanalysis,engine,structuralmechanics– Pharmaceuticals:molecularmodeling– Entertainment:fulllengthmovies(“ToyStory”)2Page2Example:ScientificComputing• MolecularDynamicsonIntelParagonwith128processors(1994)• Improveovertime:loadbalancing,other• 128processorIntelParagon=406MFLOPS• C90vector=145MFLOPS(or- 45Intelprocessors)Opportunities:CommercialComputing•Transactionprocessing&TPC-Cbenchmark– smallscaleparallelprocessorstolargescale•Others:Fileservers,electronicCADsimulation(multipleprocesses),WWWsearchenginesWhatlevelParallelism?• Bitlevelparallelism:1970to-1985– 4bits,8bit,16bit,32bitmicroprocessors• Instructionlevelparallelism(ILP):1985throughtoday– Pipelining– Superscalar– VLIW– Out-of-Orderexecution– LimitstobenefitsofILP?• ProcessLevelorThreadlevelparallelism;mainstreamforgeneralpurposecomputing– Serversareparallel– Highend DesktopdualprocessorPCFlynn’sTaxonomyofParallelArchitectures• SingleInstructionSingleData(SISD)– uniprocessor• SingleInstructionMultipleData(SIMD)– Illiac-IV,CM-2• MultipleInstructionSingleData(MISD)• MultipleInstructionMultipleData(MIMD)MIMDadvantages• MIMDsaremoreflexible– Functionassingleand/ormultiprogrammedmachine• MIMDs arecosteffective– Off-the-shelfmicroprocessors3Page3DataParallelModel(SIMD)• Operationscanbeperformedinparalleloneachelementofalargeregulardatastructure,suchasanarray• 1ControlProcessorbroadcasttomanyPEs– ConditionflagperPEsothatcanskip• Datadistributedineachmemory• DataparallelprogramminglanguageslayoutdatatoprocessorDataParallelModel(SIMD)• SIMDledtoDataParallelProgramminglanguages• AdvancingVLSIledtosinglechipFPUs andwholefastµProcs (SIMDlessattractive)• SIMDprogrammingmodelledtoSingleProgramMultipleData(SPMD)model– Allprocessorsexecuteidenticalprogram• Dataparallelprogramminglanguagesstilluseful,docommunicationallatonce:“BulkSynchronous” phasesinwhichallcommunicateafteraglobalbarrierSmall-ScaleMIMD• Memory:centralizedwithuniformaccesstime(“UMA”)andbusinterconnect• Examples: SPARCCenter,ChallengeLarge-ScaleMIMD• Memory:distributedwith nonuniform accesstime(“NUMA”)andscalableinterconnect(distributedmemory)• Examples:T3D,Exemplar,Paragon,CM-54Page4CommunicationModelsforNUMA• DistributedSharedMemory(DSM)– Processorscommunicatewithsharedaddressspace– Easyonsmall-scalemachines– Advantages:» Modelofchoicefor uniprocessors,small-scaleMPs» Easeofprogramming• Messagepassing– Processorshaveprivate memoriesseparateaddressspace– communicateviamessages– Advantages:» Lesshardware,easiertodesign» Focusesattentiononcostlynon-local operationsSharedAddressModel• Eachprocessorcannameeveryphysicallocationinthemachine• Eachprocesscannamealldataitshareswithotherprocesses• Datatransfervialoadandstore• Datasize:byte,word,...orcacheblocks• Usesvirtualmemorytomapvirtualtolocalorremotephysical• Memoryhierarchymodelapplies:nowcommunicationmovesdatatolocalproc.cache(asloadmovesdatafrommemorytocache)MessagePassingModel• Wholecomputer(CPU,memory,I/Odevices)communicateasexplicitI/Ooperations– EssentiallyNUMAbutintegratedatI/Odevicesvs.memorysystem• Sendspecifieslocalbuffer+receivingprocessonremotecomputer• Receivespecifiessendingprocessonremotecomputer+localbuffertoplacedata–


View Full Document

UMD CMSC 411 - Multiprocessors

Documents in this Course
Load more
Download Multiprocessors
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Multiprocessors and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Multiprocessors 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?