1CS 350: Introduction toSoftware EngineeringSlide Set 5Software QualityC. M. OverstreetOld Dominion UniversitySpring 2006Lecture Topics  What is quality? The economics of quality Defect-removal methods Design and code reviews Quality measures Review considerations What is Quality?  Basic definition: meeting the users’ needs needs, not wants true functional needs are often unknowable There is a hierarchy of needs that differ depending on needs Do the required tasks. Meet performance requirements. Be usable and convenient. Be economical and timely. Be dependable and reliable.2The PSP Quality Focus -1 To be useful, software must  install quickly and easily run consistently properly handle normal and abnormal cases not do destructive or unexpected things be essentially bug-free Defects are not important to users, as long as they do not affect operations cause inconvenience cost time or money cause loss of confidence in the program’s resultsThe PSP Quality Focus -2 The defect content of software products must be managed before more important quality issues can be addressed. Low defect content is an essential prerequisite to a quality software process. Since low defect content can best be achieved where the defects are injected, engineers should remove their own defects determine the causes of their defects learn to prevent those defectsThe Economics of Quality Software is the only modern technology that relies on testing to manage quality. With common quality practices, software groups typically spend 50+% of the schedule in test devote more than half their resources to fixing defects cannot predict when they will finish deliver poor-quality and over-cost products To manage cost and schedule, you must manage quality.  To get a quality product out of test, you must put a quality product into test.3Fall 2005 CS 350/ODU 7Testing Alone is IneffectiveOverloadHardware failureOperatorerrorData errorResourcecontentionConfigurationSafe and secure region = tested (shaded)Unsafe and insecure region = untested(black)Removing Defects in Test When performing a task thousands of times, economics would suggest that you use the most efficient methods. A 50,000 LOC system with traditional development methods would have 25+ defects/KLOC at test entry -1250 defects take 12,500+ programmer hours to test be late and over budget At the typical rate of 10+ hours per defect, this is 6 programmer years.9Quality and Productivity100 developers40 hours x 50 weeks200,000 hours50% test time = 100,000 hours in testManaged quality = 60% increased team productivityCurrent MethodsManaged Quality100,000 for development2 LOC/hour = 200 KLOC20% test time = 40,000 hours of test160,000 for development2 LOC/hour = 320 KLOC4Defect-removal Methods -1 The principal ways to find and fix defects are by  compiling unit testing integration and system testing team inspections personal reviews Since you will likely have to remove lots of defects, you should use the most efficient methods.Source: Xerox110100100010000Time in MinutesDesignReviewDesignInspect.CodeReviewCodeInspect.UnitTestSystemTestDefect-removal Phase522225321405Defect-removal TimesDefect-removal Methods -2 In a personal review you privately review your product your objective is to find and fix defects before test Reviews are most effective when they are structured and measured. Use reviews for requirements, test plans, code design, code, test data, and everything else that you develop. Also continue to use inspections, compiling, and testing.5Defect-removal Rates -1  Even at the personal level, it is more efficient to find defects in reviews than in testing. Unit test finds only about 2 to 4 defects per hour. Unit test finds about 50% of the defects. Code reviews find about 6 to 10 defects per hour. Practiced reviewers can find 70% or more of the defects before compiling or testing.Defect-removal Rates -205101520253012345678910Program NumberDefects Removed per HourCompileCRDLDRTest810 DevelopersWhy Reviews are Efficient In testing, you must  detect unusual behavior figure out what the test program was doing find where the problem is in the program figure out which defect could cause such behavior This can take a lot of time. With reviews and inspections, you  follow your own logic know where you are when you find a defect know what the program should do, but did not  know why this is a defect are in a better position to devise a correct fix6Review Principles PSP reviews follow a defined process with guidelines, checklists, and standards. The PSP review goal is to find every defect before the first compile or test. To address this goal, you should review before compiling or testing use coding standards use design completeness criteria measure and improve your review process use a customized personal checklistThe Code Review Checklist Your reviews will be most effective when your personal checklist is customized to your own defect experience. Use your own data to select the checklist items. Gather and analyze data on the reviews. Adjust the checklist with experience. Do the reviews on a printed listing, not on the screen. The checklist defines the review steps and the suggested order for performing them. Review for one checklist item at a time. Check off each item as you complete it.Design Review Principles  In addition to reviewing code, your should also review your designs. This requires that you  produce designs that can be reviewed• Two ways to survive a design inspection:• Produce a design that is so complex no one can understand it• Produce a design that is so simple everyone can understand it follow an explicit review strategy review the design in stages verify that the logic correctly implements the requirements7Reviewable Designs A reviewable design has a defined context precise representation consistent and clear structure This suggests that the design’s purpose and function be explicitly stated you have criteria for design completeness• typical completeness criterion: be able to pass design to programmers• (sometimes other documents and refs may be required) the design is structured in logical elementsThe Design Review Strategy