5-1©2011 Raj JainCSE567MWashington University in St. LouisThe Art of The Art of Workload SelectionWorkload SelectionRaj Jain Washington University in Saint LouisSaint Louis, MO [email protected] slides are available on-line at:http://www.cse.wustl.edu/~jain/cse567-11/5-2©2011 Raj JainCSE567MWashington University in St. LouisOverviewOverview Services Exercised Example: Timesharing Systems Example: Networks Example: Magnetic Tape Backup System Level of Detail Representativeness Timeliness Other Considerations in Workload Selection5-3©2011 Raj JainCSE567MWashington University in St. LouisThe Art of Workload SelectionThe Art of Workload SelectionConsiderations: Services exercised Level of detail Loading level Impact of other components Timeliness5-4©2011 Raj JainCSE567MWashington University in St. LouisServices ExercisedServices Exercised SUT = System Under Test CUS = Component Under Study5-5©2011 Raj JainCSE567MWashington University in St. LouisServices Exercised (Cont)Services Exercised (Cont) Do not confuse SUT w CUS Metrics depend upon SUT: MIPS is ok for two CPUs but not for two timesharing systems. Workload: depends upon the system. Examples: CPU: instructions System: Transactions Transactions not good for CPU and vice versa Two systems identical except for CPU Comparing Systems: Use transactions Comparing CPUs: Use instructions Multiple services: Exercise as complete a set of services as possible.5-6©2011 Raj JainCSE567MWashington University in St. LouisExample: Timesharing SystemsExample: Timesharing Systems Applications ⇒ Application benchmark Operating System ⇒ Synthetic Program Central Processing Unit ⇒ Instruction Mixes Arithmetic Logical Unit ⇒ Addition instruction5-7©2011 Raj JainCSE567MWashington University in St. LouisExample: NetworksExample: Networks5-8©2011 Raj JainCSE567MWashington University in St. LouisLevel of DetailLevel of Detail Most frequent request: Examples: Addition Instruction, Debit-Credit, Kernels Valid if one service is much more frequent than others Frequency of request types Examples: Instruction mixes Context sensitivity ⇒ Use set of services History-sensitive mechanisms (caching) ⇒ Context sensitivity Time-stamped sequence of requests May be too detailed Not convenient for analytical modeling May require exact reproduction of component behavior5-9©2011 Raj JainCSE567MWashington University in St. LouisLevel of Detail (Cont)Level of Detail (Cont) Average resource demand Used for analytical modeling Grouped similar services in classes Distribution of resource demands Used if variance is large Used if the distribution impacts the performance Workload used in simulation and analytical modeling: Non executable: Used in analytical/simulation modeling Executable workload: can be executed directly on a system5-10©2011 Raj JainCSE567MWashington University in St. LouisRepresentativenessRepresentativenessThe test workload and real workload should have the same: Elapsed Time Resource Demands Resource Usage Profile: Sequence and the amounts in which different resources are used.5-11©2011 Raj JainCSE567MWashington University in St. LouisTimelinessTimeliness Users are a moving target. New systems new workloads Users tend to optimize the demand. Fast multiplication Higher frequency of multiplication instructions. Important to monitor user behavior on an ongoing basis.5-12©2011 Raj JainCSE567MWashington University in St. LouisOther Considerations in Workload SelectionOther Considerations in Workload Selection Loading Level: A workload may exercise a system to its: Full capacity (best case) Beyond its capacity (worst case) At the load level observed in real workload (typical case). For procurement purposes Typical For design best to worst, all cases Impact of External Components: Do not use a workload that makes external component a bottleneck All alternatives in the system give equally good performance. Repeatability5-13©2011 Raj JainCSE567MWashington University in St. LouisSummarySummary Services exercised determine the workload Level of detail of the workload should match that of the model being used Workload should be representative of the real systems usage in recent past Loading level, impact of external components, and repeatability or other criteria in workload selection5-14©2011 Raj JainCSE567MWashington University in St. LouisExercise 5.1Exercise 5.1 What metric and workload would you choose to compare:a. Two systems with similar functionality: IBM PC versus MACb. Two systems for very different applications: PC versus Workstationsc. Two systems with identical functionality: IBM PC versus Dell PCd. Two versions of the same operating systems: Windows 98 vs Windows XPe. Two hardware components: Two floppy drivesf. Two languages: C vs. PascalOne metric and one workload is sufficient5-15©2011 Raj JainCSE567MWashington University in St. LouisExercise 5.2Exercise 5.2 Select an area of computer systems, for example, databases, networks, processors, and so on. Prepare a table identifying increasing levels of services, components, factors, and workloads.5-16©2011 Raj JainCSE567MWashington University in St. LouisHomework 5Homework 5 Read chapters 4 and 5 Submit answer to Exercise
View Full Document
Unlocking...