CS427: Software Engineering IAdministrative infoSoftware quality assurancePerspective on qualitySQATotal quality management“Quality is free”Cost of fixing an errorError: Terminology?Failure vs. flawFailure costsPrevention costsJohnson’s lawWays not to improve qualityHow to improve qualityMetricsHow to appraise qualityBug trackingSlide 19Use quality information to make decisionsSlide 21Technical reviewsMain goal: Evaluate qualitySecondary goal: Improve qualityThe review teamLeaderRecorderReviewersResult of reviewWalkthroughInspectionReviewing productsHomework 4To be reviewedTo reviewIn real lifeExample checklist: Smalltalk codeSummaryNext: DFD or UI design?1CS427:Software Engineering IDarko Marinov(slides from Ralph Johnson)CS427 20-2Administrative infoHW2 graded, get hard copies from GaneshGrades posted on WikiHW3 due todayWill grade it before the Thanksgiving breakHW4 posted, due next Thursday, Nov. 16Reviews done in pairs across different groupsCreate “pairs” of “pairs” as soon as possible Project groups start/keep meeting with TAsGet feedback on projects from TAsCS427 20-3Software quality assuranceSQA: not just testingHow can you tell if software has high quality?How can we measure the quality of software?How can we increase the quality of software?CS427 20-4Perspective on qualityCustomerSystem not crashesSystem follows documentationSystem is logical and easy to useDeveloperSystem is easy to changeSystem is easy to understandSystem is pleasant to work onCS427 20-5SQAPotential mistakesQuality is conformance to requirements and standardsVariation control is the heart of quality control (mass production unlike software)Iterative viewFeedback and continual improvement is the real heart of quality softwareCS427 20-6Total quality managementFactoriesGoal is for every item coming off the assembly line to be perfect Management, production, engineering, QAEveryone is involved in qualityDevelop a reliable, repeatable processContinuously improve the processCS427 20-7“Quality is free”QualityEffortCS427 20-8Cost of fixing an errorDesign Code Dev.TestSystemTestFieldoperation3-6timesReq.10times15-40times30-70times40-1000times101time10010001CS427 20-9Error: Terminology?AnomalyBugCrashDefectErrorFailure/fault/flaw/“feature”…CS427 20-10Failure vs. flawFailure - program didn’t work rightFlaw - mistake in the text of the programFailure analysis (debugging) - what flaw caused this failure?Flaw analysis - what is wrong with our process that allowed this flaw to be created and not detected?CS427 20-11Failure costsInternalReworkRepairFailure analysis ExternalResolving complaintsReturning and replacing productHelp lineCS427 20-12Prevention costsPreventionPlanningManaging and collecting informationReviewsAppraisalInspectionTestingCS427 20-13Johnson’s law“If you don’t test for it, your system doesn’t have it.”Is it easy to use? Easy to maintain? Does it crash?Does it match the documentation?Does it make customers happy?CS427 20-14Ways not to improve qualitySay “Be more careful!”Say “Quality is important.”Find out whose fault it is and fire himCS427 20-15How to improve qualityMeasure and compareDetermine root cause of problemsCreate ways to eliminate problemsCS427 20-16MetricsIf you don’t see it, it doesn’t existMeasure quality over time (metrics)Display in a public placeMake quality goals, then check to see if you meet themCS427 20-17How to appraise qualityRequirementsReviews by customersPrototypingAnalysis and design modelsFormal reviews, inspectionsCurrent systemBug reportsUser testsSurveysCS427 20-18Bug trackingKeep track ofWho reported the bug (the failure)Description of the failureSeverityThe flaw that caused this failureWho is repairing itThe repairCS427 20-19Bug trackingUse information about failures to estimate reliabilityCompareCritical nature of failureIteration failure discoveredModule that had the flawCS427 20-20Use quality information to make decisions“Must repair all level 1 failures before shipping”“Half of all level 1 and 2 failures in the alpha release were in the Call Processing module; we should rewrite it.”“Half of all level 1 and 2 defects found in the design reviews were in Call Processing; we should rewrite it.”CS427 20-21Bug trackingDiscover the flaw (defect) that caused each bugCategorize flawsLook at categories with the most flaws and improve your process to eliminate themCS427 20-22Technical reviewsA way to evaluate the quality of requirements, designs, and softwareA way to improve the quality of requirements, designs, and softwareA way to educate new developers and ensure that developers are consistentProven to be cost-effective!CS427 20-23Main goal: Evaluate qualityProduce a report describing Potential problemsSummary of overall qualityPass/failEvaluated by expert outsidersMust know enoughShouldn’t know too muchCS427 20-24Secondary goal:Improve qualityFind flawsEnforce standardsImprove standardsProvide feedback to managementCS427 20-25The review teamLeader (moderator)RecorderReviewersCS427 20-26LeaderResponsible for obtaining a good review - or reporting why a good review wasn’t possibleGood review - one that accurately describes the quality of the productMake sure that reviewers have all the material they need for the reviewGet a time and place for the reviewCS427 20-27RecorderResponsible to provide information for an accurate report of the reviewTypically writes notes on a “flip chart” or other public mediumAt end of review, recorder gives summary and makes sure the team agreesRecorder helps leader make final reportCS427 20-28ReviewersStudy product in advance and take notesHave a check-list of review criteriaGive both positive and negative commentsRaise issues, don’t resolve themMust be technically competentStick to standards - or stick the standardsCS427 20-29Result of reviewReview summaryWho, what, when and the conclusionIssues listCan result in more detailed reportsGive priority to issuesCan be disagreement on issuesMost issues are about product, but can also be about process or standardsCS427