Software QualitiesSoftware Quality AssuranceCosts of Poor QualityQA TerminologyTerminologyMore terminologySlide 7Slide 8Slide 9Many different kinds of testingMore kinds of testingSlide 12Testing is hard workTesting is hard work (cont.)TestingTesting PrinciplesTesting Principles (cont.)TestabilityQuality is an on-going concernDebuggingSoftware ReviewsSoftware Reviews (cont.)Typical Review TeamSoftware Review GuidelinesTechnical Review Guidelines (cont.)Sample evaluation Guidelines: Code InspectionSample evaluation Guidelines: Code Inspection (cont.)SQA SummaryCSE470: Software Engineering: QCSE470: Software Engineering: Quality Assuranceuality Assurance1Software QualitiesMaintainerUserCustomerGood DocumentationReadable CodeGood DesignLow CostPortability Increased productivityReliabilityCorrectnessEfficiencyFunctionalityEase of useEase of learningCSE470: Software Engineering: QCSE470: Software Engineering: Quality Assuranceuality Assurance2Software Quality AssuranceUse of analysis to validate artifactsrequirements, designs, code, test plansTechnical reviewsDocument reviewsCompliance to standardsControl of changesCSE470: Software Engineering: QCSE470: Software Engineering: Quality Assuranceuality Assurance3Costs of Poor QualityIncreased time to find and fix problemsIncreased cost to distribute modificationsIncreased customer supportProduct liabilityFailure in the market placeCSE470: Software Engineering: QCSE470: Software Engineering: Quality Assuranceuality Assurance4QA TerminologyCorrectnessReliabilityTestingDebuggingFailureFaultErrorVerificationValidationV&VCSE470: Software Engineering: QCSE470: Software Engineering: Quality Assuranceuality Assurance5TerminologyCorrectness: artifact is consistent with its specificationSpecification could be wrong or incomplete Rarely is software known to be correctRarely is the specification correctReliability: probability that the software is correct Statistical measure based on past performancee.g., mean time to failureCSE470: Software Engineering: QCSE470: Software Engineering: Quality Assuranceuality Assurance6More terminologyTesting: entails executing the software on selected test casesEvaluate the results (oracle)Evaluate the performanceEvaluate the ease of useCommon Wisdom: Testing reveals bugs but does not guarantee the absence of bugsHow should you select test cases?How do you know when to stop testing?CSE470: Software Engineering: QCSE470: Software Engineering: Quality Assuranceuality Assurance7More terminologyFailure: an erroneous resultincorrect outputs/response for given inputs/stimulifails to meet real-time constraintsError: incorrect conceptmay cause failures if not correctedFault: the cause of one or more failuresdiscovered after releaseCSE470: Software Engineering: QCSE470: Software Engineering: Quality Assuranceuality Assurance8More terminologyDebugging: the process of finding the cause of a “bug” and a way to fix itw/o introducing additional bugs!Verification: process of proving, using mathematical reasoning, that a program is “correct”is expensive and is not always possibleis not foolproofCSE470: Software Engineering: QCSE470: Software Engineering: Quality Assuranceuality Assurance9More terminologyValidation: process associated with showing that the software performs reasonably wellV & V: verification & validation?more typically equated with validationCSE470: Software Engineering: QCSE470: Software Engineering: Quality Assuranceuality Assurance10Many different kinds of testingUnit testing: test individual componentstest stubs simulate called componentstest harness simulates “outer” context and maintains stubsIntegration testing: combine components and test themfollows build planSystem testing: test whole systemCSE470: Software Engineering: QCSE470: Software Engineering: Quality Assuranceuality Assurance11More kinds of testingAcceptance testing: testing to determine if the product is acceptableRegression testing: retesting after the system has been modifieddetermine “old” test cases that must be re-executeddetermine what new test cases are requiredCSE470: Software Engineering: QCSE470: Software Engineering: Quality Assuranceuality Assurance12More kinds of testingBlack box / functional testing: testing based on specificationsWhite box / structural testing:testing based on looking at the artifactBoth black box and white box testing are neededCSE470: Software Engineering: QCSE470: Software Engineering: Quality Assuranceuality Assurance13Testing is hard workTypically 50% of software development effort goes into testingup to 85% for life-critical softwareHow to identify “good” test cases?high probability of finding a new errorObjective is to find errorstest case is “successful” if it finds a new errorCSE470: Software Engineering: QCSE470: Software Engineering: Quality Assuranceuality Assurance14Testing is hard work (cont.)Psychologically difficult for a programmer to test his/her own code thoroughlyExhaustive testing requires testing all combinations of input valuesSorting an array of size 10 containing integers in the range 1 . . 10 has 10! combinations (3,628,800 cases)CSE470: Software Engineering: QCSE470: Software Engineering: Quality Assuranceuality Assurance15TestingCAN:Uncover errorsShow specifications are met for specific test casesBe an indication of overall reliabilityCANNOT:Prove that a program is error-freeCSE470: Software Engineering: QCSE470: Software Engineering: Quality Assuranceuality Assurance16Testing PrinciplesTests should be traceable to requirementsTests should be planned long before testing beginsExhaustive testing is not possible80% of all errors typically occur in 20% of the modules test cases should be chosen to maximize likelihood of finding an errorCSE470: Software Engineering: QCSE470: Software Engineering: Quality Assuranceuality Assurance17Testing Principles (cont.)Testing should be done by someone other than the developersDevelopers do original testingSQA does independent testingusually black box testingAutomated testing tools should be used Reduce testing costsReduce likelihood of human errorCSE470: Software Engineering: QCSE470: Software Engineering: Quality Assuranceuality