inst eecs berkeley edu cs61c CS61C Machine Structures Lecture 42 Parallel Computing 2005 05 09 Andy Carle cs61c ta inst The California legislature is currently working on a bill to ban remote hunting via the internet after the incorporation of a Texas company specializing in a unique combination of robotics web cameras and weapons Years of Counter Strike practice and I can t even get a meal out of it CS61C L42 Parallel Computing 1 Carle Spring 2005 UCB Scientific Computing Traditional Science 1 Produce theories and designs on paper 2 Perform experiments or build systems Has become difficult expensive slow and dangerous for fields on the leading edge Computational Science Use ultra high performance computers to simulate the system we re interested in Acknowledgement Many of the concepts and some of the content of this lecture were drawn from Prof Jim Demmel s CS 267 lecture slides which can be found at http www cs berkeley edu demmel cs267 Spr05 CS61C L42 Parallel Computing 2 Carle Spring 2005 UCB Example Applications Science Global climate modeling Biology genomics protein folding drug design Astrophysical modeling Computational Chemistry Computational Material Sciences and Nanosciences Engineering Semiconductor design Earthquake and structural modeling Computation fluid dynamics airplane design Combustion engine design Crash simulation Business Financial and economic modeling Transaction processing web services and search engines Defense Nuclear weapons test by simulations Cryptography CS61C L42 Parallel Computing 3 Carle Spring 2005 UCB Performance Requirements Terminology Flop Floating point operation Flops second standard metric for expressing the computing power of a system Global Climate Modeling Divide the world into a grid e g 10 km spacing Solve fluid dynamics equations to determine what the air has done at that point every minute Requires about 100 Flops per grid point per minute This is an extremely simplified view of how the atmosphere works to be maximally effective you need to simulate many additional systems on a much finer grid CS61C L42 Parallel Computing 4 Carle Spring 2005 UCB Performance Requirements 2 Computational Requirements To keep up with real time i e simulate one minute per wall clock minute 8 Gflops sec Weather Prediction 7 days in 24 hours 56 Gflops sec Climate Prediction 50 years in 30 days 4 8 Tflops sec Climate Prediction Experimentation 50 years in 12 hours 288 Tflops sec Perspective Pentium 4 1 4GHz 1GB RAM 4x100MHz FSB 320 Mflops sec effective Climate Prediction would take 1233 years Reference http www tc cornell edu lifka Papers SC2001 pdf CS61C L42 Parallel Computing 5 Carle Spring 2005 UCB What Can We Do Wait Moore s law tells us things are getting better why not stall for the moment Parallel Computing CS61C L42 Parallel Computing 6 Carle Spring 2005 UCB Prohibitive Costs Rock s Law The cost of building a semiconductor chip fabrication plant that is capable of producing chips in line with Moore s law doubles every four years CS61C L42 Parallel Computing 7 Carle Spring 2005 UCB How fast can a serial computer be Consider a 1 Tflop sec sequential machine Data must travel some distance r to get from memory to CPU To get 1 data element per cycle this 12 means 10 times per second at the 8 m s Thus speed of light c 3x10 r c 1012 0 3 mm So all of the data we want to process must be stored within 0 3 mm of the CPU Now put 1 Tbyte of storage in a 0 3 mm x 0 3 mm area Each word occupies about 3 square Angstroms the size of a very small atom Maybe someday but it most certainly isn t going to involve transistors as we know them CS61C L42 Parallel Computing 8 Carle Spring 2005 UCB What is Parallel Computing Dividing a task among multiple processors to arrive at a unified meaningful solution For today we will focus on systems with many processors executing identical code How is this different from Multiprogramming which we ve touched on some in this course How is this different from Distributed Computing CS61C L42 Parallel Computing 9 Carle Spring 2005 UCB Recent History Parallel Computing as a field exploded in popularity in the mid 1990s This resulted in an arms race between universities research labs and governments to have the fastest supercomputer in the world Source top500 org CS61C L42 Parallel Computing 10 Carle Spring 2005 UCB Current Champions BlueGene L IBM DOE Rochester United States 32768 Processors 70 72 Tflops sec 0 7 GHz PowerPC 440 Columbia NASA Ames Mountain View United States 10160 Processors 51 87 Tflops sec 1 5 GHz SGI Altix Earth Simulator Earth Simulator Ctr Yokohama Japan 5120 Processors 35 86 Tflops sec SX6 Vector Data Source top500 org CS61C L42 Parallel Computing 11 Carle Spring 2005 UCB Administrivia HKN evaluations on Monday Last semester s final solutions online Final exam review Sunday 2005 05 08 2pm in 10 Evans Final exam Tuesday 2005 05 14 12 30 3 30pm in 220 Hearst Gym Same rules as Midterm except you get 2 double sided handwritten review sheets 1 from your midterm 1 new one green sheet Don t bring backpacks swim trunks TAs only CS61C L42 Parallel Computing 12 Carle Spring 2005 UCB Parallel Programming Processes and Synchronization Processor Layout Other Challenges Locality Finding parallelism Parallel Overhead Load Balance CS61C L42 Parallel Computing 13 Carle Spring 2005 UCB Processes We need a mechanism to intelligently split the execution of a program Fork int main int pid fork if pid 0 printf I am the child if pid 0 printf I am the parent return 0 What will this print CS61C L42 Parallel Computing 14 Carle Spring 2005 UCB Processes 2 We don t know Two potential orderings I am the child I am the parent I am the parent I am the child This situation is a simple race condition This type of problem can get far more complicated Modern parallel compilers and runtime environments hide the details of actually calling fork and moving the processes to individual processors but the complexity of synchronization remains CS61C L42 Parallel Computing 15 Carle Spring 2005 UCB Synchronization How do processors communicate with each other How do processors know when to communicate with each other How do processors know which other processor has the information they need When you are done computing which processor or processors have the answer CS61C L42 Parallel Computing 16 Carle Spring 2005 UCB Synchronization 2 Some of the logistical complexity of these operations is reduced by standard communication frameworks Message Passing
View Full Document
Unlocking...