879 CISC Parallel ComputationOutlineIntroductionBrief History of Fortran and HPFSlide 5Slide 6HPF Directives and Their SyntaxHPF-conforming or not?Programming Model of HPFData MappingSlide 11Data Mapping ExampleHPF Data Mapping ModelData Mapping ExampleHPF Data Mapping DeclarationHPF Data Mapping Example 1HPF Data Mapping Example 2HPF Data Mapping Example 3Data ParallelismSlide 20Slide 21Slide 22Slide 23Slide 24Slide 25Slide 26Putting It all TogetherExampleSlide 29Intrinsic and Library ProceduresExtrinsic ProceduresReferences and Further InformationQuestions879 CISC Parallel Computation High Performance Fortran (HPF)Ibrahim Halil [email protected] the [Fortran] group broke new ground … they never lost sight their main objective, namely to produce a product that would be acceptable to practical users with real problems to solve.Fortran … is still by far the most popular language for numerical computation Maurice V. WilkesOutlineIntroductionBrief History of Fortran and HPFHPF Directives, SyntaxData MappingData ParallelismPutting It all TogetherIntrinsic Procedures Extrinsic ProceduresReferences and Further InformationIntroductionHPF a language that combines the full Fortran 90 language with special user annotations dealing with data distribution. will be a standard programming language for computationally intensive applications on many types of machines. a set of extensions to Fortran expressing parallel execution at a high level.designed to provide a portable extension to Fortran 90 for writing data parallel applications. HPFF is the group of people who developed HPF. Since its introduction almost four decades ago, Fortran has been the language of choice for scientific and engineering programming, and HPF is the latest set of extensions to this venerable language.Brief History of Fortran and HPFEarly 1950’s The first programming language to be called Fortran was developed by IBM1957 Became popular after the first compiler delivered to a customer1966 ANSI published the first formal standard for Fortran including features likeinteger, real, double precision, do loop, if conditionals, subroutines, functions, Hollerith datatype( replaced with character type) and global Variables. This standard is called Fortran 661978 ANSI and ISO published a new standard (Fortran 77) including features likeIf then else if end if conditional statements, complex data type, complex constants, complexnumbers, character type, formatted, unformatted and direct-access file input and output.Brief History of Fortran and HPFTo satisfy the need for efficient programming on the new generation of parallelMachines, Fortran should need extensions and that leads to the beginning of HPF.1991 Desire for Revision on FORTRAN 77 standard let to the work on Fortran withthe title of Fortran 8X and resulted in 1991 by ISO and renamed as Fortran 90. Its goal wasto modernize Fortran so that it may continue its long history as a scientific and engineeringprogramming language. The first group to discuss standardization of parallel Fortran features was the ParallelComputing Forum (PCF). The original goals of the group were to standardize thelanguage features for task oriented parallelism and shared memory machines. 1991 Nov Digital Equipment Corporation organized a meeting at the Supercomputing ’91conference in Albuquerque, New Mexico to discuss HPFBrief History of Fortran and HPF1992 Jan Kickoff meeting for HPFF in Houston Texas, hosted by the Center forResearch on Parallel Computation at Rice University. Over 130 people attended and themeeting is size was larger than expected, a series of smaller “working group” meetings wasscheduled to create the language draft.1992 Mar The HPFF working group, nearly 40 people, met for the first time in Dallas,Texas. Eight further meetings were held. 1993 May The HPFF working group produced the HPF language Specification version 1.0HPF Directives and Their SyntaxThe form of an hpf-directive-line (H201) is : Directive-origin hpf-directive where a directive-origin(H202) is one of !HPF$CHPF$*HPF$Fortran 90 allows comments to begin with “C” and “*” as well as “!” in the fixed sourceform, but allows only “!” to begin a comment in free source form. There are two forms of directive in HPF: specification-directiveexecutable-diretivespecification-directive (H204): Must be in the specification part of the program unitexecutable-directive (H205): Appears with the other Fortran 90 executable-constructs in the program unit.Examples :align, distribute, processors …Examples :align, distribute, processors …HPF-conforming or not? !HPF$ DISTRIBUTE (CYCLIC) :: PERIODIC_TABLE …RIGHTREAL PERIODIC_TABLE (103); !HPF$ DISTRIBUTE PERIODIC_TABLE (CYCLIC)REAL PERIODIC_TABLE (103)!HPF$ DISTRIBUTE PERIODIC_TABLE (CYCLIC)RIGHT!HPF$ DISTRIBUTE PERIODIC_TABLE (CYCLIC); DISTRIBUTE LOG_TABLE (BLOCK)WRONG!HPF$ DISTRIBUTE PERIODIC_TABLE (CYCLIC)!HPF$ DISTRIBUTE LOG_TABLE (BLOCK)RIGHTWRONGProgramming Model of HPFProgramming ModelCommunicationParallelismFORALLDO INDEPENDENTINTRINSINC and STANDARD LIBRARY FUNCTIONSEXTRINSINC FUNCTIONSData Mapping HPF describes data-to-processor mapping by using two kind of operations:Distribute : Directive that describes how an array is divided into even-sized pieces anddistributed to processors in a regular way.REAL A (100,100) Array declarationResult : Each processor receives a 50X50 block of A, like P1 gets A(1:50,1:50)There are 4 processors in this example!HPF$ DISTRIBUTE A (BLOCK, BLOCK)Result : Each processor receives every 4.th row of A, like P1 gets A(1,1:100),A(5,1:100), A(9,1:100) ….!HPF$ DISTRIBUTE A (CYCLIC, *)Data Mapping Align : Directive that describes how two arrays ‘line up’ together. Result : X and Y are always distributed the same!HPF$ ALIGN X(I) WITH Y(I)Result : Elements of X correspond to the elements of Y(A can have at most half as many elements as Y)!HPF$ ALIGN X(I) WITH Y(2*I-1)Data Mapping Example REAL DECK_OF_CARDS (52)!HPF$ DISTRIBUTE DECK_OF_CARDS (CYCLIC)5 6 7 81 2 3 413 14 15 169 10 11 1221 22 23 2417 18 19 2029 30 31 3225 26 27 2837 38 39 4033 34 35 3645 46 47 4841 42 43 4449 50 51 522 7 12 171 6 11 164 9 14 193 8 13 1821 26 31 365 10 15 2023 28 33 3822 27 32 3725 30 35 4024 29 34 3942 47 5241 46 5143 4844 4945 50REAL DECK_OF_CARDS (52)!HPF$ DISTRIBUTE DECK_OF_CARDS (CYCLIC(5))There are 4 processors in this