DOC PREVIEW
RIT EECC 756 - Parallel Program Issues

This preview shows page 1-2-3-4-26-27-28-53-54-55-56 out of 56 pages.

Save
View full document
View full document
Premium Document
Do you want full access? Go Premium and unlock all 56 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 56 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 56 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 56 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 56 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 56 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 56 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 56 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 56 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 56 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 56 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 56 pages.
Access to all documents
Download any document
Ad free experience

Unformatted text preview:

Parallel Program IssuesParallel Programs: DefinitionsDependency Analysis & Conditions of ParallelismConditions of Parallelism: Data & Name Dependence(True) Data (or Flow) DependenceName Dependence Classification: Anti-DependenceName Dependence Classification: Output (or Write) DependenceDependency Graph ExampleSlide 9Slide 10Example Parallel Constructs: Co-begin, Co-endConditions of ParallelismBernstein’s Conditions: An ExampleAsymptotic Notations for Algorithm AnalysisSlide 15Graphs of O, W, QRate of Growth of Common Computing Time FunctionsSlide 18Theoretical Models of Parallel ComputersExample: sum algorithm on P processor PRAMExample: Sum Algorithm on P Processor PRAMPerformance of Parallel AlgorithmsMatrix Multiplication On PRAMThe Power of The PRAM ModelNetwork Model of Message-Passing MulticomputersNetwork Model of MulticomputersA Four-Dimensional (4D) HypercubeExample: Asynchronous Matrix Vector Product on a RingMatrix Vector Product y = Ax on a RingCreating a Parallel ProgramLevels of Parallelism in Program ExecutionHardware Vs. Software ParallelismComputational Parallelism and Grain SizeSlide 34Software Parallelism Types: Data Vs. Functional ParallelismExample Motivating Problem: Simulating Ocean Currents/Heat Transfer ...Limited Concurrency: Amdahl’s LawAmdahl’s Law ExampleAmdahl’s Law Example: 2-Phase n-by-n Grid Computation A Pictorial DepictionParallel Performance Metrics Degree of Parallelism (DOP)Example: Concurrency Profile of A Divide-and-Conquer AlgorithmConcurrency Profile & SpeedupAmdahl's Law with Multiple Degrees of ParallelismAmdahl's Law with Multiple Degrees of Parallelism : ExamplePictorial Depiction of ExampleParallel Performance ExampleParallel Performance Example (continued)Steps in Creating a Parallel ProgramPartitioning: Decomposition & AssignmentAssignmentOrchestrationMapping/SchedulingProgram Partitioning ExamplePowerPoint PresentationStatic Multiprocessor SchedulingSlide 56EECC756 - ShaabanEECC756 - Shaaban#1 lec # 3 Spring2008 3-20-2008Parallel Program IssuesParallel Program Issues•Dependency Analysis:Dependency Analysis:–Types of dependencyTypes of dependency–Dependency GraphsDependency Graphs–Bernstein’s Conditions of ParallelismBernstein’s Conditions of Parallelism•Asymptotic Notations for Algorithm Complexity AnalysisAsymptotic Notations for Algorithm Complexity Analysis•Parallel Random-Access Machine (PRAM)–Example: sum algorithm on P processor PRAMExample: sum algorithm on P processor PRAM•Network Model of Message-Passing MulticomputersNetwork Model of Message-Passing Multicomputers–Example: Asynchronous Matrix Vector Product on a Ring•Levels of Parallelism in Program ExecutionLevels of Parallelism in Program Execution•Hardware Vs. Software ParallelismHardware Vs. Software Parallelism•Parallel Task Grain Size Parallel Task Grain Size •Software Parallelism Types: Data Vs. Functional ParallelismSoftware Parallelism Types: Data Vs. Functional Parallelism•Example Motivating Problem With high levels of concurrencyExample Motivating Problem With high levels of concurrency•Limited Parallel Program Concurrency: Amdahl’s LawLimited Parallel Program Concurrency: Amdahl’s Law•Parallel Performance Metrics: Degree of Parallelism (DOP)Parallel Performance Metrics: Degree of Parallelism (DOP)–Concurrency ProfileConcurrency Profile•Steps in Creating a Parallel Program: Steps in Creating a Parallel Program: –Decomposition, Assignment, Orchestration, Mapping Decomposition, Assignment, Orchestration, Mapping –Program Partitioning Example (handout)Program Partitioning Example (handout)–Static Multiprocessor Scheduling Example (handout)Static Multiprocessor Scheduling Example (handout)PCA Chapter 2.1, 2.2EECC756 - ShaabanEECC756 - Shaaban#2 lec # 3 Spring2008 3-20-2008Parallel Programs: Definitions •A parallel program is comprised of a number of tasks running as threads (or processes) on a number of processing elements that cooperate/communicate as part of a single parallel computation. •Task: –Arbitrary piece of undecomposed work in parallel computation–Executed sequentially on a single processor; concurrency in parallel computation is only across tasks.•Parallel or Independent Tasks: –Tasks that with no dependencies among them and thus can run in parallel on different processing elements.•Parallel Task Grain Size: The amount of computations in a task.•Process (thread): –Abstract entity that performs the computations assigned to a task–Processes communicate and synchronize to perform their tasks•Processor or (Processing Element): –Physical computing engine on which a process executes sequentially–Processes virtualize machine to programmer•First write program in terms of processes, then map to processors•Communication to Computation Ratio (C-to-C Ratio): Represents the amount of resulting communication between tasks of a parallel programIn general, for a parallel computation, a lower C-to-C ratio is desirable and usually indicates better parallel performanceOther Parallelization Overheads CommunicationComputationParallel Execution TimeThe processor with max. execution time determines parallel execution timeEECC756 - ShaabanEECC756 - Shaaban#3 lec # 3 Spring2008 3-20-2008•Dependency analysis is concerned with detecting the presence and type of dependency between tasks that prevent tasks from being independent and from running in parallel on different processors and can be applied to tasks of any grain size.–Represented graphically as task dependency graphs.•Dependencies between tasks can be 1- algorithm/program related or 2- hardware resource/architecture related.• Algorithm/program Task Dependencies:–Data Dependence:•True Data or Flow Dependence•Name Dependence: –Anti-dependence–Output (or write) dependence–Control Dependence•Hardware/Architecture Resource DependenceDependency Analysis & Conditions of ParallelismConditions of Parallelism 12Down to task = instructionA task only executes on one processor to which it has been mapped or allocatedEECC756 - ShaabanEECC756 - Shaaban#4 lec # 3 Spring2008 3-20-2008Conditions of Parallelism: Conditions of Parallelism: Data & Name DependenceData & Name Dependence Assume task S2 follows task S1 in sequential program order1True Data or Flow Dependence: Task S2 is data dependent on task S1 if an execution path exists from S1 to S2 and if at least one output variable of S1 feeds


View Full Document

RIT EECC 756 - Parallel Program Issues

Documents in this Course
Load more
Download Parallel Program Issues
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Parallel Program Issues and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Parallel Program Issues 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?