1Falcon: On-line Monitoring and Steering of Large-Scale Parallel ProgramsFalcon: On-line Monitoring and Steering of Large-Scale Parallel ProgramsWeiming Gu, Greg Eisenhauer, Eileen Kraemer, Karsten Schwan, John Stasko, and Jeffrey VetterWeiming Gu, Greg Eisenhauer, Eileen Kraemer, Karsten Schwan, John Stasko, and Jeffrey VetterPresented by Amy Slivaz Introductionz Interactive Steeringz Contributions of Falconz Monitoring and Steering a Parallel Codez Design and Implementation of Falconz System Evaluationz Conclusions2What is interactive steering?z On-line configuration of a program by algorithms or by human users with the purpose of affecting the program’s performance or execution behaviorz Targets parallel code itselfz Rapid changes made by on-line algorithmsz Implementation of single program abstractionsz User-directed improvement of high-level attributesz Useful for understanding and improving program behaviorSo, what is Falcon?z System for on-line monitoring and steering of threads-based parallel programsz Contributions:z Application-specific monitoringz Scalable, dynamically controlled monitoring performancez On-line analysis, steering, and graphical displayz Extension to multiple heterogeneous computing platforms3z Introductionz Monitoring and Steering a Parallel Codez Requirementsz The MD Applicationz Experimentation and Resultsz Design and Implementation of Falconz System Evaluationz ConclusionsRequirements of Steeringz Application builders must write their code so steering is possiblez Users must provide the program and performance information necessary for making steering decisionsz This information must be obtainable with the latency required by the desired rate of steering4MD Applicationz Interactive molecular dynamics simulationz Simulation Process:z Obtain location information from neighborsz Calculate intra-molecular forcesz Compute inter-molecular forcesz Apply forces to yield new particle positionz Parallelism by domain decompositionSteering MD-Experimentationz Opportunities for performance improvement through on-line interactions:z Decomposition geometries changed to respond to changes in physical systemsz Modification of cutoff radius to improve solution speedz Dynamic load balancingz Local rather than global temperature calculationsz Falcon used to monitor process loads on-linez Analyze and display workload information5Steering MD-Resultsz Steering improves performance by successive adjustment of domain boundariesImportance of Resultsz Performance improves due to steering, rather than degrading because of steering costsz User interactions can be replaced by on-line steering algorithmsz Indicate the potential of on-line steering for helping users experiment with and understand the behavior of complex scientific codes6z Introductionz Monitoring and Steering a Parallel Codez Design and Implementation of Falconz Design Goalsz System Designz System Implementationz Online Steering Mechanismsz System Evaluationz ConclusionsDesign Goalsz Reduce or control monitoring latency while maintaining acceptable monitoring workloadz Supports application-specific monitoring/steering, analysis, and display of program informationz Users only capture, process, understand, and steer exactly the relevant program attributesz Support for scalable monitoring z Vary resources consumed by runtime system in accordance with machine size and program needs7System Designz Four conceptual components:z Monitoring specificaion and instrumentationz Low-level sensor specification languagez High-level view specification constructsz Instrumentation toolz Runtime libraries for information capture, collection, filtering, and analysisz Mechanisms for program steeringz Graphical user interface and displaysSteps to use Falconz Instrument application code with sensors and probesz Inserted sensors capture information during runtimez Runtime monitoring facilities collect and analyze dataz Partially processed monitoring data sent to steering mechanisms or central monitor and graphical displays8Falcon ArchitectureFalcon Implementation9System Implementationz Implemented with Cthreads libraryz Hardware platformsz Kendall Square Research KSR-1 and KSR-2 supercomputerz GP1000 BBN Butterfly multiprocessorz Sequent multiprocessorz SGI workstationsz SUN SPARC stationsSystem Implementation (cont.)z User-defined application-specific sensors capture:z Program and performance behaviors to be monitoredz Program attributes based on which steering may be performedz Sensor code can be inserted into many different places in a programz Local monitoring threads10Sensor Typesz Sampling sensorsz Write to shared memoryz Less overheadz Less detailed and accurate informationz Tracing sensorsz Generate timestamped event recordsz Extended sensorsz Performs simple analyses and filtering before outputSensor Controlz Combine sensor types to balance low monitoring latency against accuracyz Dynamically turn sensors’ instrumentation points on and offz Sensors dynamically adjust their own behaviorz Employ different monitoring mechanisms at different points in time11Concurrent Monitoringz Local monitoring and steering threads act concurrently and asynchronously with the target applicationz Can change the number of local monitoring threads and communication buffersz Adapt configuration to dynamic workload changesOn-line Steering Mechanismsz Steering serverz Separate execution threadz Receives monitoring events relevant to steeringz ‘Decide’ what actions to take using steering event databasez Steering clientz User interface and controlz Enable/Disable steering actionsz Display/Update database12On-line Steering Mechanismsz Probesz Update program attributes asynchronously to the program executionz Actuatorsz Used to enact steering actionsz Actions operate on program attributesz Execute functions to ensure that modifications do not violate correctness criteriaz Introductionz Monitoring and Steering a Parallel Codez Design and Implementation of Falconz System Evaluationz Sensor Performancez Monitoring Latency and Perturbationz Monitoring MDz Performance of On-line Steeringz Conclusions13Experimental Setupz Kendall Square Research KSR-2z 64 processors interconnected by two ringsz Non-uniform shared memory cache-only architecture z Hierarchically connected rings z Support 32 nodes eachz 64-bit processorz 32 Mbytes main memory used as local cachez 0.5 Mbyte sub-cachez Ring interfaceSensor Performancez Cost of executing each sensorz Accessing