CS 267 Applications of Parallel Computers Lecture 5: Sources of Parallelism (continued) Shared-Memory MultiprocessorsOutlineRecap: Source of Parallelism and LocalityContinuous Variables, Continuous ParametersExample: Deriving the Heat EquationExplicit Solution of the Heat EquationParallelism in Explicit Method for PDEsImplicit Solution2D Implicit MethodAlgorithms for Solving the Poisson EquationAdministrativeProgramming Recap and History of SMPsRelationship of Architecture and Programming ModelShared Address Space Programming ModelExample: shared flag indicating full/emptyHistorical Perspective60s Mainframe MultiprocessorsCaches: A Solution and New Problems70s breakthroughTechnology PerspectiveBus Bottleneck and CachesCache Coherence: The Semantic ProblemSnoopy Cache CoherenceWrite-Back/Ownership SchemesProgramming SMPsScalable Cache-Coherence90 Scalable, Cache Coherent MultiprocessorsSGI Origin 200090’s Pushing the bus to the limit: Sun Enterprise90’s Pushing the SMP to the massesCaches and Scientific ComputingScalable Global Address SpaceStructured Shared Memory: SPMDLarge Scale Shared Physical AddressCray T3DThe Cray T3DT3D Local Read (average latency)T3D Remote Read UncachedBulk Read OptionsWhere are things goingCS267 Lecture 51CS 267 Applications of Parallel ComputersLecture 5: Sources of Parallelism (continued) Shared-Memory MultiprocessorsKathy Yelickhttp://www.cs.berkeley.edu/~dmartin/cs267/CS267 Lecture 52Outline°Recap°Parallelism and Locality in PDEs•Continuous Variables Depending on Continuous Parameters•Example: The heat equation•Euler’s method•Indirect methods°Shared Memory Machines•Historical Perspective: Centralized Shared Memory•Bus-based Cache-coherent Multiprocessors•Scalable Shared Memory MachinesCS267 Lecture 53Recap: Source of Parallelism and Locality°Discrete event system•model is discrete space with discrete interactions•synchronous and asynchronous versions•parallelism over graph of entities; communication for events°Particle systems•discrete entities moving in continuous space and time•parallelism between particles; communication for interactions°ODEs•systems of lumped (discrete) variables, continuous parameters•parallelism in solving (usually sparse) linear systems•graph partitioning for parallelizing the sparse matrix computation°PDEs (today)CS267 Lecture 54Continuous Variables, Continuous ParametersExamples of such systems include°Heat flow: Temperature(position, time)°Diffusion: Concentration(position, time)°Electrostatic or Gravitational Potential:Potential(position)°Fluid flow: Velocity,Pressure,Density(position,time)°Quantum mechanics: Wave-function(position,time)°Elasticity: Stress,Strain(position,time)CS267 Lecture 55Example: Deriving the Heat Equation0 1x x+hConsider a simple problem°A bar of uniform material, insulated except at ends°Let u(x,t) be the temperature at position x at time t°Heat travels from x to x+h at rate proportional to:°As h 0, we get the heat equation:d u(x,t) (u(x-h,t)-u(x,t))/h - (u(x,t)- u(x+h,t))/h dt h= C *d u(x,t) d2 u(x,t) dt dx2= C *CS267 Lecture 56Explicit Solution of the Heat Equation°For simplicity, assume C=1°Discretize both time and position°Use finite differences with xt[i] as the heat at• time t and position I•initial conditions on x0[i]•boundary conditions on xt[0] and xt[1]°At each timestep°This corresponds to•matrix vector multiply•nearest neighbors on gridt=5t=4t=3t=2t=1t=0x[0] x[1] x[2] x[3] x[4] x[5]x[i]t+1 = z*xt[i-1] + (1-2*z)*xt[i] + z*xt[i+1]where z = k/h2CS267 Lecture 57Parallelism in Explicit Method for PDEs°Partitioning the space (x) into p largest chunks•good load balance (assuming large number of points relative to p)•minimized communication (only p chunks)°Generalizes to •multiple dimentions•arbitrary graphs (= sparse matrices)°Problem with explicit approach•numerical instability•need to make the timesteps very smallCS267 Lecture 58Implicit Solution°As with many (stiff) ODEs, need an implicit method°This turns into solving the following equation°Where I is the identity matrix and T is:°I.e., essentially solving Poisson’s equation(I + (z/2)*T) * xt+1 = (I - (z/2)*T) * xt2 -1 -1 2 -1 -1 2 -1 -1 2 -1 -1 2 -1T =CS267 Lecture 592D Implicit Method°Similar to the 1D case, but the matrix T is now°Multiplying by this matrix (as in the explicit case) issimply nearest neighbor computation°To solve this system, there are several techniques4 -1 -1-1 4 -1 -1 -1 4 -1 -1 -1 4 -1 -1-1 -1 4 -1 -1 -1 4 -1 -1 -1 4 -1 -1 -1 4T =CS267 Lecture 510Algorithms for Solving the Poisson EquationAlgorithm Serial PRAM Mem #Procs°Dense LU N3N N2N2°Band LU N2N N3/2N°Jacobi N2N N N°Conj.Grad. N 3/2N 1/2 *log N N N°RB SOR N 3/2N 1/2N N°Sparse LU N 3/2N 1/2N*log N N°FFT N*log N log N N N°Multigrid N log2 N N N°Lower bound N log N NPRAM is an idealized parallel model with zero cost communicationCS267 Lecture 511Administrative°HW2 extended to Monday, Feb. 16th°Break°On to shared memory machinesCS267 Lecture 512Programming Recap and History of SMPsCS267 Lecture 513Relationship of Architecture and Programming ModelHW / SW interface(comm. primitives)operatingsystemUser / System InterfacecompilerlibraryProgramming ModelParallel ApplicationHardwareCS267 Lecture 514Shared Address Space Programming Model°Collection of processes°Naming:•Each can name data in a private address space and•all can name all data in a common “shared” address space°Operations•Uniprocessor operations, plus sychronization operations on shared address-lock, unlock, test&set, fetch&add, ...°Operations on the shared address space appear to be performed in program order•it’s own operations appear to be in program order•all see a consistent interleaving of each other’s operations•like timesharing on a uniprocessor– explicit synchronization operations used when program ordering is not sufficient.CS267 Lecture 515Example: shared flag indicating full/empty°Intuitively clear that intention was to convey meaning by order of stores°No data dependences°Sequential compiler /
View Full Document
Unlocking...