Model based Software Testing Test Assessment and EnhancementLearning ObjectivesWhat is Test Assessment?Test Assessment (contd.)Test assessment (contd.)Slide 6Slide 7Slide 8Slide 9Test Assessment ProcedurePrinciple Underlying Test AssessmentThe Coverage PrincipleThe Coverage Principle (contd.)Slide 14Slide 15Slide 16Slide 17Slide 18Test EffortError Detection EffectivenessEffectiveness (contd.)Slide 22Effectiveness-continuedStrength of a coverage criterionThe Saturation EffectSaturation Effect: Fault ViewSaturation Effect: Reliability ViewCoverage principle-discussionControl flow graphControl flow graph (contd.)Slide 31Control flow graph-exampleControl-flow GraphStructure-based Test AdequacyStatement CoverageStatement coverage (contd.)Statement Coverage (contd.)Statement Coverage-WeaknessStatement coverage-weaknessBranch (or edge) coverageBranch coverageSlide 42Branch Coverage-WeaknessSlide 44Condition CoverageCondition coverage (contd.)Slide 47Branch coverage-weakness (contd.)Slide 49Path coveragePath coverage (contd.)Path Coverage-WeaknessPath Coverage-WeaknessesVariants of Path CoverageHierarchy in Control flow criteriaExerciseTest strategyDefinitionsDefinitions: Reliable TechniqueDefinitions: Weakly ReliableExample: Error Detection [1]Example: Error Detection (contd.)Slide 63Data flow graphdefs and usesdef-use (contd.)Slide 67Slide 68global-defdef-use graph: definitionsSlide 71def-use graph (contd.)Slide 73def-use graph exerciseSlide 75Test generationSUDS processing: Phase IATAC processing: phase IIMutation TestingMutation testing (contd.)MutantAnother mutantSlide 83Mutation scoreTest adequacy criterionMutant OperatorsMutation operatorMutation OperatorsEquivalent mutantEquivalent mutant (contd.)Mutation Testing ProcedureMutation Testing Procedure (contd.)Slide 93Tools for Mutation TestingUses of Mutation TestingSummarySummary (contd.)Model based Software Testing Test Assessment and EnhancementAditya P. MathurPurdue UniversityFall 2005Last update: August 18, 2005Software Testing and Reliability Aditya P. Mathur 20022Learning ObjectivesTo understand the relevance and importance of test assessment.To learn the fundamental principle underlying test assessment.To learn various methods and tools for test assessment.To understand the relative strengths/weaknesses of test assessment methods.To learn how to improve tests based on a test assessment procedure.Software Testing and Reliability Aditya P. Mathur 20023What is Test Assessment?Once a test set T, a collection of test inputs, has been developed, we ask:How good is T?It is the measurement of the goodness of T which is known as test assessment.Test assessment is carried out based on one or more criteria.Software Testing and Reliability Aditya P. Mathur 20024Test Assessment (contd.)These criteria are known as test adequacy criteria.Test assessment is also known as test adequacy assessment.Software Testing and Reliability Aditya P. Mathur 20025Test assessment (contd.)Test assessment provides the following information:A metric, also known as the adequacy score or coverage, usually between 0 and 1.A list of all the weaknesses found in T, which when removed, will raise the score to 1.The weaknesses depend on the criteria used for assessment.Software Testing and Reliability Aditya P. Mathur 20026Test assessment (contd.)Once the coverage has been computed, and the weaknesses identified, one can improve T.Improvement of T is done by examining one or more weaknesses and constructing new test requirements designed to overcome the weakness(es).The new test requirements lead to new test specifications and to further testing of the program.Software Testing and Reliability Aditya P. Mathur 20027Test Assessment (contd.)This is continued until all weaknesses are overcome, i.e. the adequacy criterion is satisfied (coverage=1).In some instances it may not be possible to satisfy the adequacy criteria for one or more of the following reasons:Lack of sufficient manpowerWeaknesses that cannot be removed because they are infeasible.Software Testing and Reliability Aditya P. Mathur 20028Test Assessment (contd.)The cost of removing the weaknesses is not justified.While improving T by removing its weaknesses, one usually tests the program more thoroughly than it has been tested so far.This additional testing is likely to result in the discovery of remaining errors.Software Testing and Reliability Aditya P. Mathur 20029Test Assessment (contd.)Test assessment and improvement is applicable throughout the testing process and during all stages of software development.Hence we say that test assessment and improvement helps in the improvement of software reliability.Software Testing and Reliability Aditya P. Mathur 200210Test Assessment ProcedureYesImprove TNoMeasure adequacy of Tw.r.t. C.2Is T adequate?3Yes4More testing is warranted ?5Select an adequacycriterion C.1Develop T0NoDone6Software Testing and Reliability Aditya P. Mathur 200211Principle Underlying Test AssessmentThere is a uniform principle that underlies test assessment throughout the testing process.This principle is referred to as the coverage principle.It has come about as a result of intensive research at Purdue and other research groups in software testing.Software Testing and Reliability Aditya P. Mathur 200212The Coverage PrincipleTo formulate and understand the coverage principle, we need to understand:coverage domainscoverage elementsA coverage domain is a finite domain, related to the program under test, that we want to cover. Coverage elements are the individual elements of this domainSoftware Testing and Reliability Aditya P. Mathur 200213The Coverage Principle (contd.)Coverage DomainsCoverage ElementsRequirementsClassesFunctionsInterface mutationsExceptionsSoftware Testing and Reliability Aditya P. Mathur 200214The Coverage Principle (contd.)Measuring test adequacy and improving a test set against a sequence of well defined, increasingly strong, coverage domains leads to improved confidence in the reliability of the system under test.Software Testing and Reliability Aditya P. Mathur 200215The Coverage Principle (contd.)Note the following properties of a coverage domain:It is related to the program under test.It is finite.It may come from program requirements, related to the inputs and outputs.Software Testing and
View Full Document
Unlocking...