Message Passing with PVM and MPI PVM z z Provide a simple free portable parallel environment Run on everything Parallel Hardware SMP MPPs Vector Machines Network of Workstations ATM Ethernet UNIX machines and PCs running Win32 API Works on a heterogenous collection of machines handles type conversion as needed z Provides two things message passing library point to point messages synchronization barriers reductions OS support process creation pvm spawn CMSC 818S Alan Sussman from J Hollingsworth 2 PVM Environment UNIX PVMD PVMD Sun SPARC Sun SPARC Application Process Application Process Bus Network PVMD PVMD PVMD Application Process Application Process IBM RS 6000 z DECmmp 12000 Cray Y MP Application Process One PVMD per machine all processes communicate through pvmd by default z Any number of application processes per node CMSC 818S Alan Sussman from J Hollingsworth 3 PVM Message Passing z All messages have tags an integer to identify the message defined by the user z Messages are constructed then sent pvm pk int char float var count stride pvm unpk int char float to unpack z All processes are named based on task ids tids local remote processes are the same z Primary message passing functions pvm send tid tag pvm recv tid tag CMSC 818S Alan Sussman from J Hollingsworth 4 PVM Process Control z Creating a process z pvm spawn task argv flag where ntask tids flag and where provide control of where tasks are started ntask controls how many copies are started program must be installed on target machine Ending a task pvm exit does not exit the process just the PVM machine z Info functions pvm mytid get the process task id CMSC 818S Alan Sussman from J Hollingsworth 5 PVM Group Operations z Group is the unit of communication a collection of one or more processes processes join group with pvm joingroup group name each process in the group has a unique id pvm gettid group name z Barrier can involve a subset of the processes in the group pvm barrier group name count z Reduction Operations pvm reduce void func void data int count int datatype int msgtag char group int rootinst result is returned to rootinst node does not block pre defined funcs PvmMin PvmMax PvmSum PvmProduct CMSC 818S Alan Sussman from J Hollingsworth 6 PVM Performance Issues z Messages have to go through PVMD can use direct route option to prevent this problem z Packing messages semantics imply a copy extra function call to pack messages z Heterogenous Support information is sent in machine independent format has a short circuit option for known homogenous comm passes data in native format then CMSC 818S Alan Sussman from J Hollingsworth 7 Sample PVM Program int main int argc char argv int myGroupNum int friendTid int mytid int tids 2 int message MESSAGESIZE int c i okSpawn if myGroupNum 0 Initialize the message for i 0 i MESSAGESIZE i message i 1 Now start passing the message back and forth for i 0 i ITERATIONS i if myGroupNum 0 Initialize process and spawn if necessary pvm initsend PvmDataDefault myGroupNum pvm joingroup ping pong pvm pkint message MESSAGESIZE 1 mytid pvm mytid pvm send friendTid msgid if myGroupNum 0 I am the first process pvm recv friendTid msgid pvm catchout stdout pvm upkint message MESSAGESIZE 1 okSpawn pvm spawn MYNAME argv 0 1 friendTid if okSpawn 1 else printf Can t spawn a copy of myself n pvm recv friendTid msgid pvm exit pvm upkint message MESSAGESIZE 1 exit 1 pvm initsend PvmDataDefault pvm pkint message MESSAGESIZE 1 tids 0 mytid pvm send friendTid msgid tids 1 friendTid else I am the second process friendTid pvm parent pvm exit tids 0 friendTid exit 0 tids 1 mytid pvm barrier ping pong 2 CMSC 818S Alan Sussman from J Hollingsworth 8 MPI z Goals Standardize previous message passing PVM P4 NX MPL Support copy free message passing Portable to many platforms z Features point to point messaging group collective communications profiling interface every function has a name shifted version z Buffering in standard mode no guarantee that there are buffers possible that send will block until receive is called z Delivery Order two sends from same process to same dest will arrive in order no guarantee of fairness between processes on recv CMSC 818S Alan Sussman from J Hollingsworth 9 MPI Communicators z Provide a named set of processes for communication plus a context system allocated unique tag z All processes within a communicator can be named numbered from 0 n 1 z Allows libraries to be constructed application creates communicators library uses it prevents problems with posting wildcard receives adds a communicator scope to each receive z All programs start with MPI COMM WORLD Functions for creating communicators from other communicators split duplicate etc Functions for finding out about processes within communicator size my rank CMSC 818S Alan Sussman from J Hollingsworth 10 Non Blocking Point to point Functions z Two Parts post the operation wait for results z Also includes a poll test option checks if the operation has finished z Semantics must not alter buffer while operation is pending wait returns or test returns true and data not valid for a receive until operation completes CMSC 818S Alan Sussman from J Hollingsworth 11 Collective Communication z z Communicator specifies process group to participate Various operations that may be optimized in an MPI implementation Barrier synchronization Broadcast Gather scatter with one destination or all in group Reduction operations predefined and user defined Also with one destination or all in group Scan prefix reductions z Collective operations may or may not synchronize Up to the implementation so application can t make assumptions CMSC 818S Alan Sussman from J Hollingsworth 12 MPI Misc z MPI Types All messages are typed base primitive types are pre defined int double real unsigned short char long can construct user defined types includes non contiguous data types z Processor Topologies Allows construction of Cartesian arbitrary graphs May allow some systems to run faster z z Language bindings for C Fortran C What s not in MPI 1 process creation I O one sided communication CMSC 818S Alan Sussman from J Hollingsworth 13 Sample MPI Program include mpi h int main int argc char argv int myrank friendRank char message MESSAGESIZE int i tag MSG TAG MPI Status status Now start passing the message back and forth for i 0 i ITERATIONS i if myrank 0 MPI send message MESSAGESIZE MPI CHAR friendRank tag MPI COMM WORLD MPI Recv message MESSAGESIZE Initialize no