5-1©2008 Raj JainCSE567MWashington University in St. LouisThe Art of The Art of Workload SelectionWorkload SelectionRaj Jain Washington University in Saint LouisSaint Louis, MO [email protected] slides are available on-line at:http://www.cse.wustl.edu/~jain/cse567-08/5-2©2008 Raj JainCSE567MWashington University in St. LouisOverviewOverview! Services Exercised" Example: Timesharing Systems" Example: Networks" Example: Magnetic Tape Backup System! Level of Detail! Representativeness! Timeliness! Other Considerations in Workload Selection5-3©2008 Raj JainCSE567MWashington University in St. LouisThe Art of Workload SelectionThe Art of Workload SelectionConsiderations:! Services exercised! Level of detail! Loading level! Impact of other components! Timeliness5-4©2008 Raj JainCSE567MWashington University in St. LouisServices ExercisedServices Exercised! SUT = System Under Test! CUS = Component Under Study5-5©2008 Raj JainCSE567MWashington University in St. LouisServices Exercised (Cont)Services Exercised (Cont)! Do not confuse SUT w CUS! Metrics depend upon SUT: MIPS is ok for two CPUs but not for two timesharing systems.! Workload: depends upon the system.! Examples:" CPU: instructions" System: Transactions" Transactions not good for CPU and vice versa" Two systems identical except for CPU! Comparing Systems: Use transactions! Comparing CPUs: Use instructions" Multiple services: Exercise as complete a set of services as possible.5-6©2008 Raj JainCSE567MWashington University in St. LouisExample: Timesharing SystemsExample: Timesharing Systems! Applications ⇒ Application benchmark! Operating System ⇒ Synthetic Program! Central Processing Unit ⇒ Instruction Mixes! Arithmetic Logical Unit ⇒ Addition instruction5-7©2008 Raj JainCSE567MWashington University in St. LouisExample: NetworksExample: Networks5-8©2008 Raj JainCSE567MWashington University in St. LouisExample: Magnetic Tape Backup SystemExample: Magnetic Tape Backup System! Backup System:" Services: Backup files, backup changed files, restore files, list backed-up files." Factors: File-system size, batch or background process, incremental or full backups." Metrics: Backup time, restore time." Workload: A computer system with files to be backed up. Vary frequency of backups.! Tape Data System:" Services: Read/write to the tape, read tape label, auto load tapes." Factors: Type of tape drive." Metrics: Speed, reliability, time between failures." Workload: A synthetic program generating representative tape I/O requests.5-9©2008 Raj JainCSE567MWashington University in St. LouisMagnetic Tape System (Cont)Magnetic Tape System (Cont)! Tape Drives:" Services: Read record, write record, rewind, find record, move to end of tape, move to beginning of tape." Factors: Cartridge or reel tapes, drive size." Metrics: Time for each type of service, for example, time to read record and to write record, speed (requests/time), noise, power dissipation." Workload: A synthetic program exerciser generating various types of requests in a representative manner.! Read/Write Subsystem:" Services: Read data, write data (as digital signals)." Factors: Data-encoding technique, implementation technology (CMOS, TTL, and so forth)." Metrics: Coding density, I/O bandwidth (bits per second).5-10©2008 Raj JainCSE567MWashington University in St. LouisMagnetic Tape System (Cont)Magnetic Tape System (Cont)" Workload: Read/write data streams with varying patterns of bits." Read/Write Heads:! Services: Read signal, write signal (electrical signals).! Factors: Composition, inter-head spacing, gap sizing, number of heads in parallel.! Metrics: Magnetic field strength, hysteresis.! Workload: Read/write currents of various amplitudes, tapes moving at various speeds.5-11©2008 Raj JainCSE567MWashington University in St. LouisLevel of DetailLevel of Detail! Most frequent request:" Examples: Addition Instruction, Debit-Credit, Kernels" Valid if one service is much more frequent than others! Frequency of request types" Examples: Instruction mixes" Context sensitivity ⇒ Use set of services" History-sensitive mechanisms (caching) ⇒ Context sensitivity! Time-stamped sequence of requests" May be too detailed" Not convenient for analytical modeling" May require exact reproduction of component behavior5-12©2008 Raj JainCSE567MWashington University in St. LouisLevel of Detail (Cont)Level of Detail (Cont)! Average resource demand" Used for analytical modeling" Grouped similar services in classes! Distribution of resource demands" Used if variance is large" Used if the distribution impacts the performance! Workload used in simulation and analytical modeling:" Non executable: Used in analytical/simulation modeling" Executable workload: can be executed directly on a system5-13©2008 Raj JainCSE567MWashington University in St. LouisRepresentativenessRepresentativenessThe test workload and real workload should have the same:! Elapsed Time! Resource Demands! Resource Usage Profile: Sequence and the amounts in which different resources are used.5-14©2008 Raj JainCSE567MWashington University in St. LouisTimelinessTimeliness! Users are a moving target.! New systems ⇒ new workloads! Users tend to optimize the demand.! Fast multiplication ⇒ Higher frequency of multiplication instructions.! Important to monitor user behavior on an ongoing basis.5-15©2008 Raj JainCSE567MWashington University in St. LouisOther Considerations in Workload SelectionOther Considerations in Workload Selection! Loading Level: A workload may exercise a system to its:" Full capacity (best case)" Beyond its capacity (worst case)" At the load level observed in real workload (typical case)." For procurement purposes ⇒ Typical" For design ⇒ best to worst, all cases! Impact of External Components:" Do not use a workload that makes external component a bottleneck ⇒ All alternatives in the system give equally good performance.! Repeatability5-16©2008 Raj JainCSE567MWashington University in St. LouisSummarySummary! Services exercised determine the workload ! Level of detail of the workload should match that of the model being used! Workload should be representative of the real systems usage in recent past! Loading level, impact of external components, and repeatability or other criteria in workload selection5-17©2008 Raj JainCSE567MWashington University in St. LouisExercise 5.1Exercise 5.1! What metric and workload would you choose to compare:a. Two systems with similar functionality:
View Full Document
Unlocking...