11-1©2008 Raj JainCSE567MWashington University in St. LouisRatio GamesRatio GamesRaj Jain Washington University in Saint LouisSaint Louis, MO [email protected] slides are available on-line at:http://www.cse.wustl.edu/~jain/cse567-08/11-2©2008 Raj JainCSE567MWashington University in St. LouisOverviewOverview! Ratio Game Examples! Using an Appropriate Ratio Metric! Using Relative Performance Enhancement! Ratio Games with Percentages! Ratio Games Guidelines! Numerical Conditions for Ratio Games11-3©2008 Raj JainCSE567MWashington University in St. LouisCase Study 11.1: 6502 vs. 8080Case Study 11.1: 6502 vs. 8080! Conclusion: 6502 is worse. It takes 4.7% more time than 8080.1. Ratio of Totals11-4©2008 Raj JainCSE567MWashington University in St. Louis6502 vs. 8080 (Cont)6502 vs. 8080 (Cont)1. Ratio of Totals: 6502 is worse. It takes 4.7% more time than 8080.2. With 6502 as a base: 6502 is better. It takes 1% less time than 8080.3. With 8080 as a base: 6502 is worse. It takes 6% more time.2. 6502 as the base: 3. 8080 as the base:11-5©2008 Raj JainCSE567MWashington University in St. LouisCase Study 11.2: RISC vs. CISCCase Study 11.2: RISC vs. CISC! Conclusion: RISC-I has the largest code size. The second processor Z8002 requires 9% less code than RISC-I.11-6©2008 Raj JainCSE567MWashington University in St. LouisRISC vs. CISC (Cont)RISC vs. CISC (Cont)! Conclusion: Z8002 has the largest code size and that it takes 18% more code than RISC-I. [Peterson and Sequin 1982]11-7©2008 Raj JainCSE567MWashington University in St. LouisUsing an Appropriate Ratio MetricUsing an Appropriate Ratio Metric1. Throughput: A is better2. Response Time: A is worse3. Power: A is betterExample:11-8©2008 Raj JainCSE567MWashington University in St. LouisUsing Relative Performance EnhancementUsing Relative Performance Enhancement! Example: Two floating point accelerators! Problem: Incomparable bases. Need to try both on the same machine11-9©2008 Raj JainCSE567MWashington University in St. LouisRatio Games with PercentagesRatio Games with Percentages! Example: Tests on two systems1. System B is better on both systems2. System A is better overall.System A:System B:11-10©2008 Raj JainCSE567MWashington University in St. LouisPercentages (Cont)Percentages (Cont)! Other Misuses of Percentages: " 1000% sounds more impressive than 11-time. Particularly if the performance before and after the improvement are both small" Small sample sizes disguised in percentages" Base = Initial. 400% reduction in prices ⇒ Base = Final11-11©2008 Raj JainCSE567MWashington University in St. LouisRatio Games GuidelinesRatio Games Guidelines1. If one system is better on all benchmarks, contradictingconclusions can not be drawn by any ratio game technique11-12©2008 Raj JainCSE567MWashington University in St. LouisGuidelines (cont)Guidelines (cont)2. Even if one system is better than the other on all benchmarks, a better relative performance can be shown by selecting appropriate base." In the previous example, System A is 40% better than System B using raw data, 43% better using system A as a base, and 42% better using System B as a base.3. If a system is better on some benchmarks and worse on others, contracting conclusions can be drawn in some cases. Not in all cases.4. If the performance metric is an LB metric, it is better to use your system as the base5. If the performance metric is an HB metric, it is better to use your opponent as the base6. Those benchmarks that perform better on your system should be elongated and those that perform worse should be shortened11-13©2008 Raj JainCSE567MWashington University in St. LouisNumerical Conditions for Ratio GamesNumerical Conditions for Ratio Games! A is better than B iff! A is better than B iff! Raw Data:! With A as the Base:11-14©2008 Raj JainCSE567MWashington University in St. LouisNumerical Conditions (Cont)Numerical Conditions (Cont)! A is better than B iff! With B as the base:11-15©2008 Raj JainCSE567MWashington University in St. LouisNumerical Conditions (Cont)Numerical Conditions (Cont)Ratio of B/A response on benchmark iRatio of B/A response onbenchmark j12011231Raw DataBase BBase AA isbetterusing all 3B is betterusing all 311-16©2008 Raj JainCSE567MWashington University in St. LouisSummarySummary! Ratio games arise from use of incomparable bases! Ratios may be part of the metric! Relative performance enhancements! Percentages are ratios! For HB metrics, it is better to use opponent as the base11-17©2008 Raj JainCSE567MWashington University in St. LouisExercise 11.1Exercise 11.1! The following table shows execution times of three benchmarks I, J, and K on three systems A, B, and C. Use ratio game techniques to show the superiority of various systems.11-18©2008 Raj JainCSE567MWashington University in St. LouisExercise 11.2Exercise 11.2! Derive conditions necessary for you to be able to use the technique of combined percentages to your advantage.11-19©2008 Raj JainCSE567MWashington University in St. LouisHomeworkHomework! Read chapter 11! Submit answer to Exercise
View Full Document
Unlocking...