RIT EECC 756 - Parallel Architectures History (25 pages)

Previewing pages 1, 2, 24, 25 of 25 page document View the full content.
View Full Document

Parallel Architectures History



Previewing pages 1, 2, 24, 25 of actual document.

View the full content.
View Full Document
View Full Document

Parallel Architectures History

192 views


Pages:
25
School:
Rochester Institute of Technology
Course:
Eecc 756 - Processor Systems
Processor Systems Documents
Unformatted text preview:

Parallel Architectures History Historically parallel architectures tied to programming models Divergent architectures with no predictable pattern of growth Application Software Systolic Arrays Dataflow System Software Architecture SIMD Message Passing Shared Memory EECC756 Shaaban 1 lec 2 Spring 2000 Current Trends In Parallel Architectures The extension of computer architecture to support communication and cooperation OLD Instruction Set Architecture NEW Communication Architecture Defines Critical abstractions boundaries and primitives interfaces Organizational structures that implement interfaces hardware or software Compilers libraries and OS are important bridges today EECC756 Shaaban 2 lec 2 Spring 2000 Modern Parallel Architecture Layered Framework CAD Database Multiprogramming Shared address Scientific modeling Message passing Parallel applications Data parallel Programming models Compilation or library Operating systems support Communication hardware Communication abstraction User system boundary Hardware software boundary Physical communication medium EECC756 Shaaban 3 lec 2 Spring 2000 Programming Models Programming methodology used in coding applications Specifies communication and synchronization Examples Multiprogramming No communication or synchronization at program level Shared memory address space Message passing Explicit point to point communication Data parallel More regimented global actions on data Implemented with shared address space or message passing EECC756 Shaaban 4 lec 2 Spring 2000 Communication Abstraction User level communication primitives provided Realizes the programming model Mapping exists between language primitives of programming model and these primitives Supported directly by hardware or via OS or via user software Lot of debate about what to support in software and gap between layers Today Hardware software interface tends to be flat i e complexity roughly uniform Compilers and software play important roles as bridges today Technology trends exert strong influence Result is convergence in organizational structure Relatively simple general purpose communication primitives EECC756 Shaaban 5 lec 2 Spring 2000 Communication Architecture User System Interface Implementation User System Interface Communication primitives exposed to user level by hardware and systemlevel software Implementation Organizational structures that implement the primitives hardware or OS How optimized are they How integrated into processing node Structure of network Goals Performance Broad applicability Programmability Scalability Low Cost EECC756 Shaaban 6 lec 2 Spring 2000 Toward Architectural Convergence Evolution and role of software have blurred boundary Send receive supported on SAS machines via buffers Can construct global address space on massively parallel MP message passing machines by carrying along pointers specifying the process and local virtual address space Shared virtual address space in message passing machines can also be established at the page level generating a page fault for remote pages handled by sending a message Hardware organization converging too Tighter integration even for MP low latency high bandwidth Network interface tightly integrated with memory cache controller Transfer data directly to from user address space DMA transfers across the network At lower level even hardware SAS passes hardware messages Even clusters of workstations SMPs are becoming parallel systems Emergence of fast system area networks SAN ATM fiber channel Programming models distinct but organizations converging Nodes connected by general network and communication assists Implementations also converging at least in high end machines EECC756 Shaaban 7 lec 2 Spring 2000 Convergence of Scalable Parallel Machines Generic Parallel Architecture A generic modern multiprocessor Network Communication assist CA Mem P Node processor s memory system plus communication assist Network interface and communication controller Scalable network Convergence allows lots of innovation now within framework Integration of assist with node what operations how efficiently EECC756 Shaaban 8 lec 2 Spring 2000 Understanding Parallel Architecture Traditional taxonomies not very useful Programming models are not enough nor hardware structures Can be supported by radically different architectures Architectural distinctions that affect software Compilers libraries programs Design of user system and hardware software interface Constrained from above by programming models and below by technology Guiding principles provided by layers What primitives are provided at communication abstraction How programming models map to these How they are mapped to hardware EECC756 Shaaban 9 lec 2 Spring 2000 Fundamental Design Issues At any layer interface contract aspect and performance aspects Naming How are logically shared data and or processes referenced Operations What operations are provided on these data Ordering How are accesses to data ordered and coordinated to satisfy program threads dependencies Replication How are data replicated to reduce communication overheads Communication Cost Latency bandwidth overhead occupancy Understand at programming model level first since that sets requirements from lower layers Other issues Node Granularity How to split between processors and memory EECC756 Shaaban 10 lec 2 Spring 2000 Sequential Programming Model Contract Naming Can name any variable in virtual address space Hardware and perhaps compilers does translation to physical addresses Operations Loads and Stores Ordering Sequential program order Performance Rely on dependencies on single location mostly dependence order Compilers and hardware violate other orders without getting caught Compiler reordering and register allocation Hardware out of order pipeline bypassing write buffers Transparent replication in caches EECC756 Shaaban 11 lec 2 Spring 2000 SAS Programming Model Naming Any process can name any variable in shared space Operations loads and stores plus those needed for ordering and thread synchronization Simplest Ordering Model Within a process thread sequential program order Across threads some interleaving as in time sharing Additional orders through synchronization Again compilers hardware can violate orders without getting caught Different more subtle ordering models also possible EECC756 Shaaban 12 lec 2 Spring 2000 Synchronization Mutual exclusion locks Ensure certain operations on certain data can


View Full Document

Access the best Study Guides, Lecture Notes and Practice Exams

Loading Unlocking...
Login

Join to view Parallel Architectures History and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Parallel Architectures History and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?