15-740, Fall 2007Tutorial on Using cobalt at NCSA and rachel at PSCIn the following, HOMEDIR refers to the directory:/afs/cs.cmu.edu/academic/class/15740-f07/publicand ASSTDIR refers to the subdirectory HOMEDIR/asst/asst2.1 About the MachinesWe will be using two parallel machines that support OpenMP this semester: cobalt at the National Centerfor Supercomputing Applications (NCSA), and rachel at the Pittsburgh Supercomputing Center (PSC).cobalt is a collection of SGI Altix machines, which are built using Intel Itanium 2 processors. There isa smaller cluster (co-login1) for interactive use: you will be logging onto these machines to compile andtest your code. Once your code is ready for a timing measurement, you will submit it to a batch queueto run on the 512-processor SGI Altix machine (co-compute1). There is a wealth of information aboutcobalt and how to use it at the following web site:http://www.ncsa.uiuc.edu/UserInfo/Resources/Hardware/SGIAltix/rachel is an HP AlphaServer comprised of 64 Alpha EV7 processors. Details about rachel can be foundhere:http://www.psc.edu/machines/marvel/rachel.html2 Logging InYou should use ssh to connect to the front-end machines at both NCSA and PSC. In particular, type thefollowing to log onto the interactive cobalt machine (co-login1) at NCSA:> ssh [email protected] the following to log into rachel:> ssh [email protected] Compiling your programsBecause the OpenMP s tandard is supported on both cobalt and rachel, you should not need to modifyyour source code to run on either platform. Since these machines were designed by different vendorsand have different processors, compilers, etc., you do need to compile your code with different com-pilers and different compiler flags. You can find several trivial OpenMP C code examples in ASST-DIR/examples, along with a Makefile for each machine: i.e. ASSTDIR/examples/Makefile.ncsa andASSTDIR/examples/Makefile.psc.13.1 Compiling on cobaltTo compile OpenMP programs on cobalt, use icc and specify -openmp to the compiler. For example, tocompile a program hello.c with optimization, type:> icc -openmp -O hello.c -o hello3.2 Compiling on rachelTo compile OpenMP programs on rachel, use cc and specify -omp to the compiler. Because the frontendnode of rachel have different processors (EV6.7) than the compute node (EV7). You need to tell thecompiler to optimize for the compute processors by specifying -O -fast -tune ev7 -arch ev7. Forexample, to compile a program hello.c with optimization, type:> cc -omp -O -fast -tune ev7 -arch7 hello.c -o hello4 Running your programsFor both cobalt and rachel, you can run your program on the frontend nodes (i.e. the ones that youlogin to) by simply typing the program name at the command line directly (i.e. the way that you wouldnormally run a program on a UNIX-based system.) This is a very useful way to debug your program, andto get some rough performance numbers.To get a proper performance measurement on either machine, however, you need to submit your programto the batch queueing systems. When your program reaches the head of the batch queue, it will run withthe machine (or your portion of the machine) to itself.4.1 Submitting batch jobs on cobaltTo learn how to submit batch jobs on cobalt, please read the following web page:http://www.ncsa.uiuc.edu/UserInfo/Resources/Hardware/SGIAltix/Doc/Jobs.html4.2 Submitting batch jobs on rachelrachel uses the Portable Batch Scheduler (PBS) system to control acce ss to compute processors for batchjobs. Therefore you need to compose different job scripts for rachel. Please see the following web pagefor details:http://www.psc.edu/machines/marvel/rachel.html#batchIn particular, search for the sample script that follows “A sample job script for an OpenMP program is”.You can also learn more about PBS by typing:> man pbs2You may have noticed that there is no explicit mem ory size specification in this job sc ript. This is becauseRachel always associates 3.7 GB memory with each processor. Therefore, when the job script requests 8processors, 3.7GB * 8 = 29.6GB memory is allocated for the job
View Full Document
Unlocking...