PARALLEL & CLUSTER COMPUTINGCS 6260PROFESSOR: ELISE DE DONCKERBY: LINA HUSSEIN1 Introduction What is cluster computing? Classification of Cluster ComputingTechnologies:Technologies: Beowulf cluster Construction of Beowulf Cluster The use of cluster computing in Bioinformatics & Parallel Computing Folding@Home Project High performance clusters (HPC)a 256-processor Sun cluster. Build Your Own Cluster!2 Mainly in parallel: Split problem in smaller tasks that are executed concurrently Why? Absolute physical limits of hardware components  Economical reasons – more complex = more expensive Performance limits – double frequency <> double performanceperformance Large applications – demand too much memory & time Advantages: Increasing speed & optimizing resources utilization Disadvantages: Complex programming models – difficult development3 Several applications on parallel processing:ScienceComputationDigital BiologyAerospaceResources Exploration4 Architectures of Parallel Computer: PVP (Parallel Vector Processor) SMP (Symmetric Multiprocessor) MPP (Massively Parallel Processor) COW (Cluster of Workstation) DSM (Distributed Shared Memory) Towards Inexpensive Supercomputing: Cluster Computing is the Commodity Supercomputing 58.8%Architectures and Technology Trend of SupercomputerCluster 58.80%MPP 20.00%Other5 A computer cluster is a group of linked computers, workingtogether closely so that in many respects they form a singlecomputer. The components of a cluster are commonly, but notalways, connected to each other through fast local area networks.Clusters are usually deployed to improve performance and/oravailability over that provided by a single computer, while typicallybeing much more cost-effective than single computers of comparablespeed or availability.Clusterconsistsof:CPU CPU CPU…High Speed Local Network Cluster Middle ware Cluster…Clusterconsistsof: Nodes(master+computing) Network OS Cluster middleware:Middleware such as MPIwhich permits computeclustering programs to beportable to a wide varietyof clusters6High availability clusters (HA) (Linux)Mission critical applicationsNetwork Load balancing clustersoperate by distributing a workload evenly over multiple back end nodes.Science ClustersHigh-availability clusters (also known as Failover Clusters) are implemented for the purpose of improving the availability of services which the cluster provides. provide redundancyeliminate single points of failure.Web servers, mail servers,..nodes.Typically the cluster will be configured with multiple redundant load-balancing front ends. all available servers process requests.Beowulf7 A Beowulf Cluster is a computer design that uses parallelprocessing across multiple computers to create cheap andpowerful supercomputers. A Beowulf Cluster in practice isusually a collection of generic computers, either stocksystems or wholesale parts purchased independently andassembled, connected through an internal network.Aclusterhastwotypesofcomputers,amastercomputer,Aclusterhastwotypesofcomputers,amastercomputer,and node computers. When a large problem or set of data isgiven to a Beowulf cluster, the master computer first runs aprogram that breaks the problem into small discrete pieces;it then sends a piece to each node to compute. As nodesfinish their tasks, the master computer continually sendsmore pieces to them until the entire problem has beencomputed.8( Ethernet,Myrinet….) + (MPI)( Ethernet,Myrinet….) + (MPI) Master: or service node or front node ( used to interact with users and manage the cluster ) Nodes : a group of computers (computing node s)( keyboard, mouse, floppy, video…) Communications between nodes on an interconnect network platform ( Ethernet, Myrinet….) In order for the master and node computers to communicate, some sort message passing control structure is required. MPI,(Message Passing Interface) is the most commonly used such control.9 To construct Beowulf cluster there are four distinct but interrelated areas of consideration:Hard ware system structureresource administration and management structuremanagement environmentDistributed Programming libraries and toolsParallel algorithms10Brief Technical Parameters: OS: CentOS 5 managed by Rochs-cluster Service node: 1 (Intel P4 2.4 GHz) Computing nodes: 32 (Intel P4 2.4- 2.8 GHz) System Memory: 1 GB per node Network Platforms: Gigabit Ethernet, 2 cards per nodeMyrinet 2 G Language: C, C++, Fortran, java  Compiler: GNU gcc, Intel compiler, sun Java compilerParallel Environment: MPICH  Tools: Ganglia (Monitoring)Pbs - Torque (Scheduler)11OS (Operating System ) Three of the most commonly used OS are including kernel level support for parallel programming: Windows NT/2000mainly used to build a High Availability Cluster or a NLB(Network Local Balance) Cluster, provide services such as NLB(Network Local Balance) Cluster, provide services such as Database , File/Print,Web,Stream Media .Support 2-4 SMP or 32 processors. Hardly used to build a Science Computing Cluster Redhat LinuxThe most used OS for a Beowulf Cluster.provides High Performance and Scalability / High Reliability / Low Cost ( get freely and uses inexpensive commodity hardware )  SUN Solaris Uses expensive and unpopular hardware12Network Platform Some design considerations for the interconnect network are:• Fast Ethernet (100Mbps): low cost / min latency: 80µs• Gigabit Ethernet (1Gbps) expensive/ min latency: 300 µs•Myrinet(high-speed local area networking system)(2Gbps) The •Myrinet(high-speed local area networking system)(2Gbps) The best network platform.• Some design considerations for the interconnect network are:• Network structure Bus/Switched• Maximum bandwidth• Minimum latency13Parallel Environment Two of the most commonly used Parallel Interface Libraries:o PVM (Parallel Virtual Machine)o MPI (Message Passing Interface) Parallel Interface Libraries: provide a group of communication interface libraries that support message passing. Users can call these libraries directly in their Fortran and C programs. Cluster Computer Architecture141516171819 HPC platform for scientific applications Storage and processing of large data Satellites image processing Information Retrieval, Data Mining Computing systems in