National Aeronautics and Space Administration Jet Propulsion Laboratory CSCI 578 Course Project Rearchitecting Scientific Code March 31 2009 David Woollard NASA Jet Propulsion Lab 4800 Oak Grove Drive Pasadena CA 91108 Dept of Computer Science University of Southern California Los Angeles CA 90089 National Aeronautics and Space Administration Jet Propulsion Laboratory Agenda For Today Introduction Scientific Computing Domain Workflows Grid Computing Research Challenge Project Goals Project Details Extracting Kernels Executable Creation Workflow Specification SWSA A Domain Specific Software Architecture Timeline CSCI 578 Course Project March 31 2009 2 National Aeronautics and Space Administration Jet Propulsion Laboratory Computational Sciences Today Real science is now being conducted via computational experimentation or in silico computing CSCI 578 Course Project March 31 2009 3 National Aeronautics and Space Administration Jet Propulsion Laboratory in silico Computing Discovery is a phase is which a scientist rapidly prototypes tests hypotheses and develops a methodology Theory Development Execution Lone Researcher Kepner 03 Discovery Practice Productio n CSCI 578 Course Project March 31 2009 Distributi on 4 National Aeronautics and Space Administration Jet Propulsion Laboratory in silico Computing Production is the engineering of replicating an experiment on large volumes of data We will focus on Production Systems in this research Discovery Productio n CSCI 578 Course Project March 31 2009 Distributi on 5 National Aeronautics and Space Administration Jet Propulsion Laboratory in silico Computing Distribution is a phase in which data is dispersed to peers for review and further experimentation including Papers Federated Data Digital Libraries Discovery Productio n CSCI 578 Course Project March 31 2009 Distributi on 6 National Aeronautics and Space Administration Jet Propulsion Laboratory Validating Computational Science Computational science like all science requires validation Validation comes in two forms Scaling in data and computation Independent replication Both forms require significant computational resources Grid is a promising resource CSCI 578 Course Project March 31 2009 7 National Aeronautics and Space Administration Jet Propulsion Laboratory Vision of the Grid Like the power grid the computational Grid should scale to the demands of individual users CSCI 578 Course Project March 31 2009 8 National Aeronautics and Space Administration Jet Propulsion Laboratory Workflow Based Specification Workflows orchestrate processes on the Grid Workflows are a processing model that incorporate tasks data and rules rules Workflow management systems execute tasks on the Grid using data once the task s dependencies are satisfied based on rules rules Task 1 Task 2 Task 3 Task 5 Task 4 CSCI 578 Course Project March 31 2009 9 National Aeronautics and Space Administration Jet Propulsion Laboratory A Plethora of Workflow Languages Yu Buyya presented a taxonomy Yu Buyya 05 Based on workflow properties like model representation and scheduling policy Illustration of divergence in the field CSCI 578 Course Project March 31 2009 Considered a Grand Challenge Gil et al 07 10 National Aeronautics and Space Administration Jet Propulsion Laboratory Scaling the Experiment T a s k 1 T a s k 2 T T a a s s k k 3 4 Other Institutions T a s k 5 Home Laboratory Institution CSCI 578 Course Project March 31 2009 Co laboratory 11 National Aeronautics and Space Administration Jet Propulsion Laboratory Independent Replication T a s k 1 T a s k 2 T T a a s s k k 3 4 T a s k 5 Collaborator CSCI 578 Course Project March 31 2009 3rd Party 12 National Aeronautics and Space Administration Jet Propulsion Laboratory Heterogeneous Environments Laboratory T a s k 1 T a s k 2 T a T s a k s k 3 Institution T a s k T a s k 5 1 Workflow Engine 1 T a s k 2 T a T s a k s k 3 Co laboratory T a s k T a s k 5 1 Workflow Engine 1 T a s k 2 T a T s a k s k 3 T a s k 5 Workflow Engine 2 4 4 4 Grid Infrastructure 1 Grid Infrastructure 2 Grid Infrastructure 2 Collaborator CSCI 578 Course Project March 31 2009 3rd Party 13 National Aeronautics and Space Administration Jet Propulsion Laboratory Research Challenge Scientific validation requires Scaling Replication Existing technologies exhibit three challenges Require scientists to become engineers or vice versa Existing workflow specifications entwine scientific and engineering concerns Existing workflow specifications are not portable CSCI 578 Course Project March 31 2009 14 National Aeronautics and Space Administration Jet Propulsion Laboratory Agenda Introduction Scientific Computing Domain Workflows Grid Computing Research Challenge Project Goals Project Details Extracting Kernels Executable Creation Workflow Specification SWSA A Domain Specific Software Architecture Timeline CSCI 578 Course Project March 31 2009 15 National Aeronautics and Space Administration Jet Propulsion Laboratory Project Goals What do you get out of it Experience as a software architect in an unfamiliar domain Experience refactoring applying architectural recovery Exposure to popular Grid technologies Experience implementing distributed software systems with architectural middleware A grade What do I get out of it Data on kernel extraction for a canonical set of scientific applications Performance impacts of architected code Feedback on methodologies CSCI 578 Course Project March 31 2009 16 National Aeronautics and Space Administration Jet Propulsion Laboratory Agenda Introduction Scientific Computing Domain Workflows Grid Computing Research Challenge Project Goals Project Details Extracting Kernels Executable Creation Workflow Specification SWSA A Domain Specific Software Architecture Timeline CSCI 578 Course Project March 31 2009 17 National Aeronautics and Space Administration Jet Propulsion Laboratory Scientific Software Packages 1 3 Search The search benchmark solves a game of connect4 on a 6 x 7 board using a alpha beta pruning technique The problem size is determined by the initial position from which the game in analyzed Euler The Euler benchmark solves the time dependent Euler equations for flow in a channel with a bump on one of the walls A structured irregular N x 4N mesh is employed and the solution method is a finite volume scheme using a fourth order Runge Kutta method with both second and fourth order damping CSCI 578 Course Project March 31 2009 18 National Aeronautics and Space