CPE 619 MonitorsPart II: Measurement Techniques and ToolsOutlineMonitorsSlide 5TerminologyTerminology (cont’d)Monitor ClassificationSlide 9Software MonitorsIssues in Software Monitor Design - Activation MechanismIssues in Software Monitor Design – Buffer SizeIssues in Software Monitor Design – Number of BuffersIssues in Software Monitor Design – Buffer OverflowIssues in Software Monitor Design – MiscSlide 16Hardware MonitorsSoftware vs. Hardware MonitorsFirmware and Hybrid MonitorsSlide 20Monitoring Distributed SystemsLayered ViewComponents of a Distributed Systems MonitorObservation (1 of 2)Observation (2 of 2)CollectionAnalysisPresentation (1 of 2)Presentation (2 of 2)Interpretation and ConsoleReal-World ExamplesPerformance TuningMeasuring Execution TimeUsing the date CommandUsing the time CommandUsing the clock() FunctionUsing the gettimeofday() functionProgram ProfilingExample: gprofPAPI InterfaceTuning ExampleTuning Example (cont’d)Slide 43CPE 619MonitorsAleksandar MilenkovićThe LaCASA LaboratoryElectrical and Computer Engineering DepartmentThe University of Alabama in Huntsvillehttp://www.ece.uah.edu/~milenkahttp://www.ece.uah.edu/~lacasa2Part II: Measurement Techniques and ToolsMeasurements are not to provide numbers but insight - Ingrid BucherMeasure computer system performanceMonitor the system that is being subjected to a particular workloadHow to select appropriate workloadIn general performance analysis should know1. What are the different types of workloads?2. Which workloads are commonly used by other analysts?3. How are the appropriate workload types selected?4. How is the measured workload data summarized?5. How is the system performance monitored?6. How can the desired workload be placed on the system in a controlled manner?7. How are the results of the evaluation presented?3OutlineIntroductionTerminologySoftware MonitorsHardware MonitorsMonitoring Distributed Systems4MonitorsA monitor is a tool used to observe activities on a systemObserve performanceCollect performance statisticsMay analyze the dataMay display resultsMay even suggest remediesMonitors are used not only by performance analystsSystems programmer may profile softwareSystem manager may measure resource utilization to find bottleneckSystem manager may use to tune systemSystem analyst may use to characterize workloadSystem analyst may use to develop models or inputs for modelsThat which is monitored improves. – Source unknown5OutlineIntroductionTerminologySoftware MonitorsHardware MonitorsMonitoring Distributed Systems6TerminologyEvent – a change in the system stateE.g.: cache miss, page fault, process context switch, beginning of seek on a disk, arrival of a packet, Trace – a log of events, usually including the time of the event, and other important parametersOverhead – most monitors perturb the system operationUse CPU or storage; Sometimes called artifact. Goal is to minimize artifactDomain – the set of activities observable by the monitorE.g.: accounting logs record information about CPU time, number of disks, terminals, networks, paging I/O’s, the number of characters transferred among disks, terminals, networks, and paging devices7Terminology (cont’d)Input rate – the maximum frequency of events that monitor can correctly observeBurst mode: the rate at which an event can occur for a short period of timeSustained mode: the rate the monitor can tolerate for long durationsResolution – coarseness of the information observedInput width – the number of bits recorded for each event. Input rate x width = storage required8Monitor ClassificationImplementation levelSoftware, Hardware, Firmware, HybridTrigger mechanismEvent driven – activated only by occurrence of certain events; Low overhead for rare event, but higher if event is frequentSampling (timer driven) – activated at fixed time intervals by clock interruptsIdeal for frequent eventsDisplayOn-line – provide data continuously. E.g.: tcpdumpBatch – collect data for later analysis. E.g.: gprof.9OutlineIntroductionTerminologySoftware MonitorsHardware MonitorsMonitoring Distributed Systems10Software MonitorsMonitor operating systems, and higher level software, e.g., networks, databasesAt each activation, several instructions are executedIn general, only suitable for low frequency event or overhead becomes too highOverhead may be OK if timing does not need to be preserved; Lower input rates, lower resolutions, and higher overhead than hardwareBut, they have higher input widths, higher recording capacitiesEasier to develop and modify11Issues in Software Monitor Design - Activation MechanismHow to trigger the data collection routine?1) Trap – instrument the system software with trap instructions at appropriate points. Collect data. Like a subroutine.E.g.: to measure I/O service time, trap before I/O service routine and record time, trap after, take diff2) Trace – each instruction is followed by data collection routine (trace mode). Enormous overhead. Time insensitive.E.g., instruction-trace monitor to produce a PC histogram3) Timer interrupt – a timer interrupt service provided by the OS is used to transfer control to a data collection routine at fixed intervals. Overhead is independent of the event rateIf sampling counter, beware of overflows12Issues in Software Monitor Design – Buffer SizeStore recorded data in buffers in memory, which are later written to hard diskBuffers should be large To minimize the need to write frequently to hard disk Buffers should be smallDon’t have a lot of overhead when write to diskDoesn’t impact performance of system (or reduced memory availability is not observable)Optimal buffer size is a function of the input rate, input width, and emptying rate13Issues in Software Monitor Design – Number of BuffersUsually organized in a ringAllows recording (buffer-emptying) process to proceed at a different rate than monitoring (buffer-filling) processMonitoring may be burstySince cannot read while process is writing, a minimum of two buffers required for concurrent accessMay be circular for writing so monitor overwrites last if recording process too slowMay compress to reduce space, but adds overhead14Issues in Software Monitor Design –