Overview Message Passing and MPI CS 594 Understanding Parallel Architectures From Theory To Practice Dr Graham E Fagg Covers message passing and the MPI standard and API history of message passing Covers more advanced features of MP systems and MPI and why they exist Examines some of the current issues in using these systems and what the future holds multi protocol systems intro to metacomputing scheduling and cluster computing issues CS 594 Message Passing MPI Lectures 2 Lecture 2 Message Passing What is Message Passing History of Message Passing Overview of MPI Including some numeric program examples How do we use MPI From compiling to fixing mistakes and getting it really running Getting help Contact the TA first TA invisible man Contacting me Email me first fagg cs utk edu 320 Claxton Phone 4 5790 Best time to find me late afternoons and evenings CS 594 Message Passing MPI CS 594 Message Passing MPI Message Passing Message Passing What is message passing Its where tasks or processes communicate via explicit send and receive operations on fixed items of data As opposed to Shared memory where normal read write operations can be used to SHARE data CS 594 Message Passing MPI Data can be passed between processes on the same machine or between processes on different machines They might not even exist at the same time I e no need for temporal coupling but this is unusual CS 594 Message Passing MPI 1 Message Passing Why message pass So we can break problems into smaller pieces for faster performance DMMP MIMD For fault tolerance multiple servers Explicit operations can be reasoned about in a formal way I e CSP Message Passing can be standardized to allow for highly portable applications Was not always true Is now happening to shared memory see OpenMP Message Passing Many computer systems cannot share resources in a transparent way Even we can we can implement message passing on top of that shared resource This sometimes yield great performance increases Message passing allows systems that avoid contention on resources Compare the IBM SP 2 to the SGI Origin2000 CS 594 Message Passing MPI CS 594 Message Passing MPI Message Passing Basics How long has it been around Since before networks Copy data known as a message onto a removable media such a a removable disk Pass the message I e move the disk across the room building country Read the message mount the disk and read it Not that silly an example until recently some universities in NYcity passed data via motorbike couriers on Exbyte tapes Until ATM has finally caught up in BandWidth We need two or more entities A sender A receiver or receivers Some data the message Some means of passing the data network shared resource The two basic operations Send and Receive And a means of identifying each entity and the message maybe CS 594 Message Passing MPI CS 594 Message Passing MPI Basics The history of message passing as we know of it The Sender passed the message by sending it ARPA net users passed message via protocols such as IP send data somewhere args The receiver gets it by asking for it receiving the message E mail is a message passing system receive some buffer some other args The variations will be examined in a few slide time CS 594 Message Passing MPI But we are interested in the true parallel computing versions CS 594 Message Passing MPI 2 Past message passing systems How the facilities and functionality of modern message passing systems evolved has been influenced by a vast number of research projects and commercial implementations by vendors of MPP machines We will briefly discuss some of these systems in terms of which features we now think of as standard that they introduced and what they omitted This will also help you understand how implementers of such systems have learned to insulate users more effectively from the increasing complexity of the underlying systems Vendor Message Passing Systems and Machines First we will cover the hardware systems and their message passing systems that led to today s range of systems Then we will cover some portable message passing systems Not an exhaustive list but long enough to show just how varied it can be CS 594 Message Passing MPI CS 594 Message Passing MPI Caltech Hypercube Caltech Hypercube The Caltech Hypercube circa 1984 was a d dimensional hypercube structured system with a computational node at the end of each vertex and a single host to control the machine known as an Intermediate Host The system was programmed in either C or Fortran77 and communication was based on a subroutine library known as the Crystalline Operating System CROS The communications library assigned addresses to tasks depending on which node they were physically located processes could only communicate to neighbors or the intermediate host a total of d 1 links The CROS terminology for a link between two nodes was that of a channel though which 8 byte message packets could be sent The system only supported collective operations broadcast to from the intermediate host and the overall communication pattern was SIMD in nature I e all processes had to call the same communications routine at the same time This lead the machine to appear halfway between the earlier SIMD machines such as the ICL DAP and Thinking Machines CM 1 2 and the later Intel iPSC and XPS Paragon machines The former where program execution and communication occurs fully in lockstep and the latter where ordering was completely independent On the Hypercube under CROS the program was free to run independently to each other but the hardware forced all the communication into lockstep For solving very regular problems in physics such as partial derivatives for large numbers of grid points the structure imposed by the programming environment was an aid for producing low level very efficient implementations CS 594 Message Passing MPI CS 594 Message Passing MPI Caltech Hypercube Caltech Hypercube This was a difficult target for programming any but the most highly regular problems This led to the development of the Distributed Process environment a kernel based small operating system which allowed greater control and flexibility than CROS In particular the user developing message passing programs no longer had to think about a fixed mapping between nodes running programs that communicate in a fixed way via channels i e hardware links but could now think about processes connected via virtual communication channels the basic address independence from
View Full Document
Unlocking...