CPS 303 High Performance ComputingWensheng ShenDepartment of Computational ScienceSUNY BrockportChapter 3: Basic MPI A sample MPI program The execution of an MPI program MPI detail Summary3.1. A sample MPI program Each process sends a greeting to another process How to identify each process? The processes are ranked from 0, 1, …, p-1, if there are p processes.  We want all processes except process 0 to send a greeting to process 0, and have it print them out.The “hello world” program#include <stdio.h>#include <string.h>#include “mpi.h”main(int argc, char *argv[]) {int my_rank; /* rank of process */Int p; /* number of processes */Int source; /* rank of sender */Int dest; /* rank of receiver */Int tag; /* tag for message */char message[100]; /* storage for message */MPI_Status status; /* receive *//* start up MPI */MPI_Init(&argc, &argv);/* find out process rank */MPI_Comm_rank(MPI_COMM_WORLD, &my_rank);/* find out number of processes */MPI_Comm_size(MPI_COMM_WORLD, &p);If(my_rank != 0) {sprintf(message, “Hello World from process %d!”, my_rank);dest = 0;MPI_Send(message, strlen(message)+1, MPI_CHAR, dest, tag, MPI_COMM_WORLD);} else {for (source = 1; source<p; source++) {MPI_Recv(message, 100, MPI_CHAR, source, tag, MPI_COMM_WORLD, &status);printf(“%s\n”, message);}}MPI_Finalize();} What is the difference between printf and sprintf? sprintf: a function that puts together a string, output goes to an array of char instead of stdoutprintf: prints to stdoutStructure of program 3.1 Library inclusion Variable declaration Start MPI Get process rank Get the number of processes If (I am not process 0) Send message Else For i=1, p-1 Receive message endfor Endif Shut down MPIExecution sequences: Assume that we run one process on each processor, the program is executed as: (1) the user issues a directive to the operating system that has the effect of placing a copy of the executable program on each processor; (2) Each processor begins execution of its copy of the executable; (3) Different processes can execute different statements by branching within the program based on their process ranks. In the most general form of MIMD programming, each process runs a different program. In practice, the goal of “each process running a different program” is obtained by putting branching statements within a single program.  Even though the statements executed by process 0 are essentiallydifferent from those executed by the other processes, we avoid writing several distinct programs by including the branching statement.If (my_rank != 0)I earn more money;ElseI earn less money;SPMD (single-program multiple-data) programming3.2 Execution % cc –o greetings hello.c –lmpi Makefile, or makefile What is the result if we run the program with 2 processes? What is the result if we run the program with 6 processes?3.3 MPI Detail Why MPI MPI is not a new language It is simply a library definitions and functions  Most of the programs are conventional C and Fortran statements Extra work: a collection of special definitions and functions3.3.1 General MPI program Every MPI program must contain the preprocessor directive All MPI identifiers begin with the string “MPI_.” # include “mpi.h” MPI_Init should be called once and only once before we can call other library functions. The parameters of MPI_Init are pointers to the main function’s parameters, argc and argv. MPI_Finalize must be called to clean up any unfinished work ---It frees memory allocation by MPI.  MPI_Init must be called before any other MPI function is called. MPI_Finalize must be called somewhere following the last call to any other MPI function.3.3.2 Process ranking in MPI In a SPMD program, the flow of control depends on the rank of a process. MPI provides the function MPI_Comm_rank, which returns the rank of a process in its second parameter. The first parameter is a communicator.  MPI_Comm_rank(MPI_COMM_WORLD, &myId); Determining the number of processes MPI_Comm_size(MPI_COMM_WORLD, &mySize);3.3.3 Message: Data + envelop How Adam sends a letter to Eve?  Sending: Adam (1) composing the letter, (2) putting the letter in the mailbox, (3) addressing and stamping the envelop, (4) dropping the letter in the mailbox, (5) the postal service collects the mail in the mailbox and delivers the letter to Eve.  Receiving: Eve (1) checks her mail, (2) finds the letter, (3) opens it, (4) read Adam’s message.Process-process communication Process A wants to send message to process B, A and B are connected by wires.  Adam and Eve and connected through the postal service. The actual message passing in our program is carried by the MPI functions MI_Send, sending a message to a designated process, and MPI_Recv, receiving a message. To send a message to B, A must: (1) compose the message, and put it in a buffer  (2) call MPI_Send trying to drop the message in a mailbox (3) enclosing the message in an envelop and adding an address (4) adding the size of the message to identify the end of message (data are a sequence of signals). Enveloping: the message is included in an envelop which contains the destination, size of message.How does B deal with mails in his mailbox? Junk mail, personal mail from acquaintance, personal mail from strangers, how does he know? B can tell this by adding the source address to each mail. In MPI, we do this by specifying the source address of the receiving message. If both junk mail and personal mail are from a single source, how can B tell? In MPI, if A sends both floats to be printed and stored, how can B distinguish them?Proposals  (1) each process sends two messages. The first specifies whether the float is to be printed or stored, and the second contains the actual float; (2) each process can send a single message, a string, that contains both the float and whether the float is to be printed or stored.Problems: To the first proposal Very expansive, communication twice. The first message may be separated from the second one, it is difficult to pair them. To the second proposal The sending process has to encode the data into string, and the receiving process has to decode the string into data It is time-consuming, and there may be a loss of precision when the float is encoded as a string and decode