Message Passing Programming Deals with parallel programming by passing messages among processing nodes and processes Several message passing have been created in recent years with most of the ideas developed merged into the PVM and MPI standards Message Passing Interface MPI A standard specification for a library of message passing functions developed by the MPI Forum Achieves portability using public domain platform independent messagepassing library Not self contained relies on underlying platform for process management Parallel Virtual Machine PVM Originally developed to enable a heterogeneous network of UNIX computers to be used as a large scale message passing parallel computer Using PVM a virtual machine a set of fully connected nodes is constructed with dynamic process creation and management Each node can be a uniprocessor or a parallel computer PVM provides a portable self contained software system with library routines to support interprocess communication and other functions EECC756 Shaaban 1 lec 6 Spring2000 3 28 2000 Process Creation In Message Passing Possible methods of generating processes 1 Static process creation In the static process creation the processes are specified before the program is executed and the system will execute afixed number of processes The programmer usually explicitly identifies the processes 2 Dynamic process creation In the dynamic process creation processes can be created and started for execution during the execution of the main program using process creation constructs or system calls processes can also be destroyed Process creation and destruction might be done conditionally The number of processes may vary during execution Clearly dynamic process creation and destruction is a more powerful technique than static process creation but it does incur very significant overheads when the processes are created EECC756 Shaaban 2 lec 6 Spring2000 3 28 2000 Process Creation In Message Passing EECC756 Shaaban 3 lec 6 Spring2000 3 28 2000 Message Passing Modes Synchronous Message Passing Process X executing a synchronous send to process Y has to wait until process Y has executed a synchronous receive from X Blocking Send Receive A blocking send is executed when a process reaches it without waiting for a corresponding receive Returns when the message is sent A blocking receive is executed when a process reaches it and only returns after the message has been received Non Blocking Send Receive A non blocking send is executed when reached by the process without waiting for a corresponding receive A non blocking receive is executed when a process reaches it without waiting for a corresponding send EECC756 Shaaban 4 lec 6 Spring2000 3 28 2000 Generic Message Passing Routines Send and receive message passing procedure system calls often have the form send parameters recv parameters where the parameters identify the source and destination processes and the data EECC756 Shaaban 5 lec 6 Spring2000 3 28 2000 Blocking send and recv System Calls EECC756 Shaaban 6 lec 6 Spring2000 3 28 2000 Non blocking send and recv System Calls EECC756 Shaaban 7 lec 6 Spring2000 3 28 2000 Parallel Virtual Machine PVM History PVM is a software environment that permits a heterogeneous collection of serial parallel and vector computers on a network to appear as a single large parallel computer The PVM project began in the summer of 1989 at Oak Ridge National Laboratory ORNL The prototype system PVM 1 0 was constructed by Vaidy Sunderam and Al Geist as a byproduct of ORNL s Heterogeneous Distributed Computing research project This version was only used internally at ORNL Version 2 of PVM was written at the University of Tennessee and was released in March 1991 After user feedback and a number of changes PVM 2 1 2 4 a complete rewrite was undertaken and version 3 was completed in February 1993 Version 3 4 released in 1998 adding Communication Context safe messages Message Handlers for extending features Persistent messages tuple space model User defined tracing for specialized tools Windows 9x NT Port EECC756 Shaaban 8 lec 6 Spring2000 3 28 2000 Major Features of PVM Resource Management Add delete hosts from a virtual machine VM Process Control Spawn kill tasks dynamically Message Passing Nonblocking send blocking and nonblocking receive Dynamic Task Groups A task can join or leave a group at any time Fault Tolerance VM automatically detects faults and adjusts accordingly EECC756 Shaaban 9 lec 6 Spring2000 3 28 2000 PVM The PVM system consists of two parts A PVM daemon pvmd that resides on every computer of the virtual machine A user callable library libpvm3 a linked to user applications for process management message passing and virtual machine management PVM Console Invoking pvm host file creates a PVM console interactive stand alone PVM process with a pvm prompt and starts a single master pvmd daemon on the invoking host and a slave pvmd daemon on every host listed in an optional host file forming the virtual machine Only the master daemon can add or delete slaves via rsh or rexec and may do so at the request of other processes in the virtual machine Dynamic Virtual Machine Configuration A user application calling PVM library functions is accomplished using pvm addhosts and the pvm delhosts functions sent to the master pvmd Host table A data structure residing in every host For each host in the virtual machine a host descriptor in the host table holds its configuration information EECC756 Shaaban 10 lec 6 Spring2000 3 28 2000 PVM Components The PVM System is composed of Pvmd3 daemon program that Runs on each host of the virtual machine Provides inter host point of contact Authenticates tasks Executes processes on the hosts Provides fault tolerance Is mainly a message router but is also a source and sink of messages libpvm programming library that is linked with each application component program Contains the PVM message passing functions Application components Written by users in PVM message passing calls Executed as PVM tasks EECC756 Shaaban 11 lec 6 Spring2000 3 28 2000 PVM Components pvmd One PVM daemon per host libpvm Tasks linked to PVM library EECC756 Shaaban 12 lec 6 Spring2000 3 28 2000 Parallel Virtual Machine PVM In PVM the problem is decomposed into separate tasks by the programmer Each program is written in C or Fortran containing embedded calls to the PVM library and compiled to run on specific types of computers in the network Process based computation The unit