Parallel Processing (CS 730) Lecture 2: Designing and Implementing Parallel Programs with Linda*IntroductionOrdered or Linked Data StructuresImplementing Distributed ListsUnbounded BufferImplementing Streams in LindaSlide 7Bounded BufferSolution using a CounterSolution using SemaphoresReaders/Writers ProblemA Solution to the Readers/Writers ProblemReaders/Writers SolutionDebugging Linda ProgramsLogical Issues in Debugging Parallel ProgramsSlide 16TuplescopeSnapshot of TuplescopeFinding all Primes between 1 and nResult Parallel ProgramAgenda Parallel ProgramSpecialist Parallel ProgramPerformance of the Specialist ProgramOct. 2, 2002 Parallel Processing 1Parallel Processing (CS 730) Lecture 2: Designing and Implementing Parallel Programs with Linda*Jeremy R. Johnson*This lecture was derived from chapters 4 and 5 in Carriero and GelernterOct. 2, 2002 Parallel Processing 2Introduction•Objective: To implement result, agenda, and specialist parallel algorithms using Linda. To further study the use of Linda as a coordination language. To develop techniques for debugging.–Exercises in coordination of programs using Linda•Bounded Buffer•Readers/Writers–Debugging parallel programs•Non-determinism•Deadlock•Tuplescope–An example: finding all primes between 1 and n•Result parallel program•Agenda parallel program•Specialist parallel programOct. 2, 2002 Parallel Processing 3Ordered or Linked Data Structures•Instead of linking by address, we link by logical name•A list cell linking A and B–Suppose C is a two element array [“A”, “B”], then the cons cell whose first element (car) is “A” and next element (cdr) is “B” could be represented by the tuple:–(“C”, “cons”, cell)–If the cell “A” is an atom we might represent it by the tuple:–(“A”, atom, value)ACBOct. 2, 2002 Parallel Processing 4Implementing Distributed Lists•Represent cons cells by–(listid, “cons”, carid, cdrid)–listid is a unique positive integer–the null list has id equal to 0–list id’s are obtained from the (“next listid”, val)–val initialized to 1 and is incremented as id’s are allocated•Represent atoms by–(atomid, “atom”, value)–atomid is a unique negative integer–value is an integer–atom id’s are obtained from the tuple (“next atomid”, val)–val is initialized to -1 and is decremented as id’s are allocatedOct. 2, 2002 Parallel Processing 5Unbounded Buffer•Problem: Two sets of processors called producers and consumers share a common buffer. –Producers put items into the buffer–Consumers remove items from the buffer–If the buffer is empty consumers must block•Solution using streams–Use a multi-source, multi-sink stream–This preserves order in which elements are produced–If producers are sufficiently ahead of the consumers the amount of items in the buffer may become arbitrarily largeOct. 2, 2002 Parallel Processing 6Implementing Streams in Linda•Sequence of elements represented by a series of tuples:–(“stream”, 1, val1)–(“stream”, 2, val2)–…•Index of the last element is kept in a tail-tuple–(“stream”, “tail”, 14)•To append–in(“stream”, “tail”, ?index)–out(“stream”, “tail”, index+1)–out(“stream”, index, NewElement)Oct. 2, 2002 Parallel Processing 7Implementing Streams in Linda•An in-stream needs a head tuple to store the index of the head value (next value to be removed)•To remove the head tuple:–in(“stream”, “head”, ? index);–out(“stream”, “head”, index+1);–in(“stream”, index, ? Element);•When the stream is empty, blocked processes will continue in the order in which they blockedOct. 2, 2002 Parallel Processing 8Bounded Buffer•Same as unbounded buffer problem except the size of the buffer is fixed•In this case if the buffer is full producers must block•The buffer can be implemented using a bag or distributed table or stream where the tail and head can differ by no more than the buffer size.•Two approaches for coordination are provided–use a tuple to count the number of elements in the buffer–use semaphores to count the number of empty and full slots in the bufferOct. 2, 2002 Parallel Processing 9Solution using a Counterint producer(int id){ int n; while (TRUE) { in("count",?n); if (n < BUF_SIZE) { out("count",n+1); out("buffer",id); } else out("count",n); }return 1;} int consumer(void){ int n; int item; while (TRUE) { in("count",?n); if (n > 0) { out("count",n-1); in("buffer",?item); printf("Consumed item from Producer %d\n",item); } else out("count",n); }return 1;}Oct. 2, 2002 Parallel Processing 10Solution using Semaphoresint producer(int id){while (TRUE) { in("empty"); out("buffer",id); out("full"); }return 1;} int consumer(int id){int item;while (TRUE) { in("full"); in("buffer",?item); printf("Consumer %d consumed item %d\n",id,item); out("empty"); }return 1;}Oct. 2, 2002 Parallel Processing 11Readers/Writers Problem•Many processes share a distributed data structure. Processes are allowed direct access to the data structure but only after they have been given permission.•Many readers or a single writer may have access to the data structure but not both.•A constant stream of read requests may not be allowed to postpone a write request and similarly a constant stream of write requests may not be allowed to postpone a read request.Oct. 2, 2002 Parallel Processing 12A Solution to the Readers/Writers Problem•Use a queue of requests - requests are handled in the order they are made. This handles starvation issues.•If the queue’s head is a read request, the request is permitted as soon as there are no writers. If the head is a write request, the request is permitted as soon as there are no readers or writers. When the request is granted it is removed from the head, reads/writes, and notifies the system when it is done.•Readers/writers determine on their own when it is permissible to proceedOct. 2, 2002 Parallel Processing 13Readers/Writers Solution•Instead of creating a distributed queue use 4 shared variables–rw-tail - requests added here–rw-head - requests serviced here–active-readers–active-writersstartread(); startread() <read> {stopread(); rd(“rw-head”,incr(“rw-tail”)); rd(“active-writers”,0); incr(“active-readers”); incr(“rw-head”);