Conventional Computer Architecture Abstraction Conventional Sequential or Single Processor Conventional computer architecture has two aspects Single 1 The definition of critical abstraction layers Processor The user system boundary What is done in user space and what support is provided by the operating system to user programs The hardware software boundary Instruction Set Architecture ISA 2 Realization of abstraction layers The organizational structures that realize implement the abstraction layers to deliver high performance in a costeffective manner Implementation of abstraction layers in system software OS hardware Conventional Sequential or Single Processor PCA Chapter 1 2 1 3 EECC756 Shaaban 1 lec 2 Spring 2008 3 13 2008 Conventional Computer Architecture Abstraction Critical Abstraction Layers System Space Software User Space Sequential User Applications Sequential Programming Model Compliers Libraries Assemblers User System Boundary Operating Systems Support Hardware Hardware Software Boundary ISA CPU System Design Implementation Conventional Sequential or Single Processor EECC756 Shaaban 2 lec 2 Spring 2008 3 13 2008 Parallel Programming Models A parallel computer system is a collection of communicating processing elements that communicate and cooperate to solve large problems fast A parallel program consists of two or more threads of control parallel tasks that operate on data Each task only executes on one processor to which it has been mapped or allocated A parallel programming model is the conceptualization of the parallel machine and programming methodology used in coding parallel applications that specifies communication and synchronization Parallel programming models specify how parallel tasks of a parallel program communicate and what synchronization operations are available to coordinate their activities and order This includes specifying 1 What data can be named by a task or thread 2 What operations can be performed on the named data 3 What order exists among these operations Naming Operations Order Typically the parallel programming model is supported at the user level by parallel languages or parallel programming environments in the form of user level communication and synchronization primitives Historically parallel architectures were tied to parallel programming models As parallel programming environments have matured it led to the separation between parallel programming models and parallel machine organization system implementation forming the communication abstraction EECC756 Shaaban 3 lec 2 Spring 2008 3 13 2008 Common Parallel Programming Models Parallel Programming Model definition Parallel programming methodology used in coding parallel applications that specifies communication and synchronization Or Most Common A parallel programming model is the conceptualization of the parallel machine and programming methodology used in coding parallel applications and specifies how parallel tasks of a parallel program communicate and what synchronization operations are available Shared memory Address Space SAS Parallel program threads or tasks communicate using a shared memory address space shared data in memory Message passing Explicit point to point communication is used between parallel program tasks using messages Data parallel More regimented global actions on data i e the same operations over all elements on an array or vector Can be and usually implemented with shared address space SAS or message passing EECC756 Shaaban 4 lec 2 Spring 2008 3 13 2008 Parallel Architectures History Historically parallel architectures and implementations were tied to programming models Divergent architectures with no predictable pattern of growth Application Software Systolic Arrays System Software Architecture SIMD Message Passing Dataflow Shared Memory As parallel programming environments have matured it led to the separation between parallel programming models and parallel machine organization system implementation extending conventional computer architecture abstraction and forming the communication abstraction PCA Chapter 1 2 1 3 EECC756 Shaaban 5 lec 2 Spring 2008 3 13 2008 Current Trends In Parallel Architectures Abstraction As defined earlier a parallel computer is a collection of processing elements that communicate and cooperate to solve large problems fast This requires the extension of conventional computer architecture abstraction user system ISA to account for communication and cooperation among processors The extension of computer architecture to support communication and cooperation OLD Instruction Set Architecture NEW Communication Architecture The Communication Architecture Defines 1 Critical abstractions boundaries Communication Abstraction Basic user level communication and synchronization operations Primitives that are used to realize a parallel programming model User System Boundary Also in conventional computer architecture abstraction Software Hardware Boundary Organizational structures that implement interfaces hardware or software 2 Compilers libraries and OS are important bridges today between programming model requirements and parallel hardware implementation EECC756 Shaaban 6 lec 2 Spring 2008 3 13 2008 Modern Parallel Architecture Abstraction Software Layered Framework CAD Database Multiprogramming Shared address Scientific modeling Message passing Parallel applications Data parallel Programming models User Space Compilation or library Hardware Operating systems support Communication hardware Communication abstraction User system boundary System Space Hardware software boundary ISA Physical communication medium Hardware Processing Nodes Interconnects EECC756 Shaaban 7 lec 2 Spring 2008 3 13 2008 Communication Abstraction The communication abstraction forms the key interface between the programming model and system implementation Plays a role in parallel architecture similar to instruction set ISA in sequential computer architecture User level communication synchronization primitives provided Realizes the parallel programming model Mapping exists between language primitives of programming model and these primitives Primitives supported directly by hardware or via OS or via user software Lot of debate about what to support in software and gap between layers Today Even for conventional computer architecture Hardware software interface tends to be flat i e complexity roughly uniform Compilers and software play important roles as bridges Technology