Course InformationContact InformationObjectivesWhat Will This Course Teach You?What Won’t This Course Teach You?Who Should Take This CourseCourse MaterialsGradingProjectSuitable Subjects for ProjectsPossible ProjectsProject ProposalExperimental DesignProject PresentationGrading of ProjectsEvaluation of Projects by Other GroupsCourse InformationAndy WangCIS 5930-03Computer Systems Performance Analysis2Contact Information•Andy Wang ([email protected])•Office: 264 Love Building•Office hours: M 4 - 5pm F 4 – 5pm (also by appointments)•Class website: http://www.cs.fsu.edu/~awang/courses/cis5930_s2009Objectives•Become exposed to common technical issues involved in evaluating software systems •Learn proper statistical tools and experimental design techniques•Gain experience in performance evaluation34What Will This CourseTeach You?•Proper methods for designing and performing experiments on system software•Proper methods for analyzing and presenting data gathered from such experiments•Proper methods for critiquing experiments and data produced by others5What Won’t This Course Teach You?•Basic systems software principles•Systems software modeling•Queueing theory•Simulation techniques for systems software6Who Should TakeThis CourseWell, everyone, but especially -•Software developers•Software researchers•Software purchasers•Software evaluatorsCourse Materials•Lecture notes and papers (posted on the class website)•Textbook–The Art of Computer Systems Performance Analysis, by Raj Jain8Grading•Up to 2 exams - 10% each•Up to 5 assignments - 5% each•Evaluation of other projects - 5%•Project – remaining gradeProject•You need to develop a project either on your own or in teams of two or three•Goal:–Publishable results10Suitable Subjectsfor Projects•Operating systems or their components (file systems, I/O subsystems, process handling, etc.)•Compilers•Databases•Real-time applications•Large application packages (e.g., email, web tools)•Networks/networking systems•The WebPossible Projects•Measure/simulate and compare systems•Develop and validate a simulator•Characterize workloads of a system•Analyze the effects of various factors on a system•Develop a software monitor to observe the performance of a system11Project Proposal•Due on the 5th week•10-minute presentation–All team members are required to participate•2-page written proposal–System background and evaluation goals–Metrics–Parameters, factors, levels–Evaluation technique–Workloads–Division of laborsExperimental Design•Due on the 10th week•10-minute presentation–All team members are required to participate•2-page written design–Brief background–Experimental design–Data analysis techniques–Anticipated graphsProject Presentation•During the last two weeks of the course•15-20 minutes•15-page (max) written paper due by the last lecture (double column, single-space, 10-pt font)15Grading of ProjectsSeveral criteria will be used:•Proper design of the experiment•Care and thoroughness of its execution•Completeness of analysis•Quality of data presentation•Insight gained from experimentInsight is the most important16Evaluation of Projectsby Other Groups•Submitted by each student individually•1 page critique on two other group’s group’s proposed experiment –Due by 11th week•1 page critique on two other group’s results–Due by final’s week•Graded on basis of insight into strengths and flaws of each