DOC PREVIEW
RIT EECC 756 - Parallel Architectures History

This preview shows page 1-2-24-25 out of 25 pages.

Save
View full document
View full document
Premium Document
Do you want full access? Go Premium and unlock all 25 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 25 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 25 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 25 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 25 pages.
Access to all documents
Download any document
Ad free experience

Unformatted text preview:

EECC756 - Shaaban#1 lec # 2 Spring 2002 3-14-2002Parallel Architectures HistoryParallel Architectures HistoryApplication SoftwareSystem SoftwareSIMDMessage PassingShared MemoryDataflowSystolicArraysArchitectureHistorically, parallel architectures tied to programming models • Divergent architectures, with no predictable pattern of growth.EECC756 - Shaaban#2 lec # 2 Spring 2002 3-14-2002Current Trends In Current Trends In Parallel ArchitecturesParallel Architectures• The extension of “computer architecture” to supportcommunication and cooperation:– OLD: Instruction Set Architecture.– NEW: Communication Architecture.• Defines:– Critical abstractions, boundaries, and primitives(interfaces).– Organizational structures that implement interfaces(hardware or software).• Compilers, libraries and OS are important bridges todayEECC756 - Shaaban#3 lec # 2 Spring 2002 3-14-2002Modern Parallel ArchitectureModern Parallel ArchitectureLayered FrameworkLayered FrameworkCADMultiprogramming SharedaddressMessagepassingDataparallelDatabase Scientific modelingParallel applicationsProgramming modelsCommunication abstractionUser/system boundaryCompilationor libraryOperating systems supportCommunication hardwarePhysical communication mediumHardware/software boundaryEECC756 - Shaaban#4 lec # 2 Spring 2002 3-14-2002Programming ModelsProgramming Models• Programming methodology used in coding applications• Specifies communication and synchronization• Examples:– Multiprogramming: No communication or synchronization at program level– Shared memory address space:– Message passing: Explicit point to point communication– Data parallel: More regimented, global actions on data• Implemented with shared address space or message passingEECC756 - Shaaban#5 lec # 2 Spring 2002 3-14-2002Communication AbstractionCommunication Abstraction• User-level communication primitives provided– Realizes the programming model.– Mapping exists between language primitives of programming modeland these primitives• Supported directly by hardware, or via OS, or via user software.• Lot of debate about what to support in software and gap betweenlayers.• Today:– Hardware/software interface tends to be flat, i.e. complexity roughlyuniform.– Compilers and software play important roles as bridges today.– Technology trends exert strong influence• Result is convergence in organizational structure– Relatively simple, general purpose communication primitives.EECC756 - Shaaban#6 lec # 2 Spring 2002 3-14-2002Communication ArchitectureCommunication Architecture = User/System Interface + Implementation• User/System Interface:– Communication primitives exposed to user-level by hardware andsystem-level software.• Implementation:– Organizational structures that implement the primitives: hardwareor OS.– How optimized are they? How integrated into processing node?– Structure of network.• Goals:– Performance– Broad applicability– Programmability– Scalability– Low CostEECC756 - Shaaban#7 lec # 2 Spring 2002 3-14-2002Toward Architectural ConvergenceToward Architectural Convergence• Evolution and role of software have blurred boundary:– Send/receive supported on SAS machines via buffers.– Can construct global address space on massively parallel (MP) message-passingmachines by carrying along pointers specifying the process and local virtualaddress space.– Shared virtual address space in message-passing machines can also beestablished at the page level generating a page fault for remote pages handledby sending a message.• Hardware organization converging too:– Tighter integration even for MP (low-latency, high-bandwidth):• Network interface tightly integrated with memory/cache controller.• Transfer data directly to/from user address space.• DMA transfers across the network.– At lower level, even hardware SAS passes hardware messages.• Even clusters of workstations/SMPs are becoming parallel systems:– Emergence of fast system area networks (SAN): ATM, fiber channel ...• Programming models distinct, but organizations converging:– Nodes connected by general network and communication assists.– Implementations also converging, at least in high-end machines.EECC756 - Shaaban#8 lec # 2 Spring 2002 3-14-2002Convergence of Scalable Parallel Machines:Convergence of Scalable Parallel Machines: Generic Parallel ArchitectureGeneric Parallel Architecture• A generic modern multiprocessor:Node: processor(s), memory system, plus communication assist:• Network interface and communication controller.• Scalable network:• Convergence allows lots of innovation, now within framework• Integration of assist with node, what operations, how efficiently...Mem° ° °NetworkP$Communicationassist (CA)EECC756 - Shaaban#9 lec # 2 Spring 2002 3-14-2002Understanding Parallel ArchitectureUnderstanding Parallel Architecture• Traditional taxonomies not very useful.• Programming models are not enough, nor hardwarestructures.– Can be supported by radically different architectures.• Architectural distinctions that affect software– Compilers, libraries, programs.• Design of user/system and hardware/software interface– Constrained from above by programming models and belowby technology.• Guiding principles provided by layers.– What primitives are provided at communication abstraction.– How programming models map to these.– How they are mapped to hardware.EECC756 - Shaaban#10 lec # 2 Spring 2002 3-14-2002Fundamental Design IssuesFundamental Design Issues• At any layer, interface (contract) aspect and performanceaspects:– Naming: How are logically shared data and/or processesreferenced?– Operations: What operations are provided on these data.– Ordering: How are accesses to data ordered andcoordinated to satisfy program threads dependencies?– Replication: How are data replicated to reducecommunication overheads?– Communication Cost: Latency, bandwidth, overhead,occupancy.• Understand at programming model level first, since thatsets requirements from lower layers.• Other issues:– Node Granularity: How to split between processors and memory?– ...EECC756 - Shaaban#11 lec # 2 Spring 2002 3-14-2002Sequential Programming ModelSequential Programming ModelContract– Naming: Can name any variable in virtual address space• Hardware


View Full Document

RIT EECC 756 - Parallel Architectures History

Documents in this Course
Load more
Download Parallel Architectures History
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Parallel Architectures History and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Parallel Architectures History 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?