I. INTRODUCTIONII. PARALLEL COMPUTINGIII. PARALLEL COMPUTER MEMORY ARCHITECTURESIV. parallel programming modelsV. the four challenges VI. conclusionParallelism: The Future of Computing? Winston A. Sumalia CS 252 [EE298] University of the Philippines Abstract—The advent of parallelism is being widely embraced by avant-garde computing groups coming from the industry and the academe. Various universities had been integrating parallel computing courses into its academic program, starting from the undergraduate degree towards the graduate level. Parallel computing per se is a broad topic, with many kinds of approaches to it. However, the main objective of this paper is to break down parallelism into some of its implementations and eventually coming up with an evaluation coming from industry and academic sources. This paper gives an overview on why parallel computing is used, comparisons between its memory architectures and programming models, and addresses some issues coming from but not limited to the 1989 paper entitled “Parallel Computing – One Opportunity Four Challenges” [ ]1. That after 17 years, programmability within the parallel architecture remains an issue especially for the industry. This is a midterm research paper for the UC-Berkeley CS252, a graduate computer architecture class, which deals greatly with parallel processing education. I. INTRODUCTION Exponential developments with respect to chip technology have been consistent for the past few decades. Computer performance has been driven largely by decreasing the size of chips while increasing the number of transistors they contain. In accordance with Moore's law, this has caused chip speeds to rise and prices to drop. This ongoing trend has driven much of the computing industry for years. Traditionally, software had been written in a serial/sequential fashion. A problem is broken into a discrete series of instructions. Instructions are executed one after another and only one instruction may execute at any moment in time. Improvement in computer performance was implemented through clock rate ramping in order to provide faster execution of the instructions; somewhat equivalent to having better instruction throughput. However, transistors can't shrink forever. Increasing clock speeds causes transistors to switch faster and thus generate more heat and consume more power. Even now, as transistor components grow thinner, chip manufacturers have struggled to cap power usage and heat generation, two critical problems. Current transistor technology limits the ability to continue making single processor cores more powerful. The transition from serial to parallel computing is one in a way to extend Moore’s law into getting more performance out of a piece of silicon. II. PARALLEL COMPUTING Parallel computer architecture had been a classic research topic for the past few decades. With the sudden sea change in computer trends, parallel computing now becomes one of the promising and interesting research thrusts geared into improving computational capability. Parallel computing is basically a development from serial computing. 2Parallel computing, in its simplest sense, is the simultaneous use of multiple compute resources to solve a computational problem. It is usually used in running multiple CPUs, enabling a problem broken down to discrete parts that it would be solved concurrently. Each part is further broken down to a series of instructions, and those instructions execute simultaneously on the different CPUs. Why use parallel computing? Aside from the limitations set for the silicon process, parallelism is aimed at saving time and performance. Being able to perform multiple tasks at any moment clearly makes a difference. In the case of parallel supercomputers, the use of multiple cost-efficient processors to work on the same task is better than relying on a single yet expensive supercomputer. The Fig. 1 clearly states that parallel computing is still in its research and development state (50% + 24%). Meanwhile, the industry (17%) is notably close in adapting it towards its commercialization, being evident in recent computer products brought about by major companies such as Intel and AMD. Fig. 1 Parallel Computing User Distribution [2] Despite its growing popularity, there have always been debates regarding the different approaches towards optimal parallelism, which deals in both hardware and software design.III. PARALLEL COMPUTER MEMORY ARCHITECTURES The way processors communicate is dependent upon memory architecture, which in turn would affect the way one would write a parallel program. 2 primary approaches in computer memory architecture involve the shared and the distributed memory model. In a shared memory, the use of a global physical memory equally shared among processors allows simple data to share through a uniform read and write mechanism into the memory. Problems arising with this model are usually due to increased contention and longer latencies, thus limits its performance and scalability. Examples of this are the Cray C-90 and Cray Y-MP. Distributed memory architectures, on the other hand, have local memory modules interconnected through a network. Their scaleable nature makes it possible for higher computing power. However, problem lies on the communication between nodes leading to complex software problems. Examples in the industry are the IBM SP1 and SP2 and the Intel Paragon. A relatively new concept involves the combining of the advantages of the two, the Distributed Shared Memory (DSM). DSM allows processes to assume globally shared virtual memory. [ ]3In Fig. 2, the distributed shared memory system consists of N networked workstations, each with its own memory. The distributed shared memory system provides the concept of a globally shared memory. Each processor can access any data items without the programmer having to worry about where the data is or how to obtain its value. Therefore the programmer does not have to specify inter processor communication which would be very complex especially for sophisticated parallelization strategies. The largest and fastest computers today employ the shared and distributed memory architecture. IV. PARALLEL PROGRAMMING MODELS There are various ways in programming parallel computers, the first two being the more popular models: Message Passing - the user makes calls to libraries to
View Full Document
Unlocking...