Experimentation in Software Engineering:Reading Studies II Scenario-Based Reading DefinitionAn approach to generating a family of reading techniques, calledoperational scenarios, has been defined to be - document and notation specific - tailorable to the project and environment- procedurally defined- goal driven - focused to provide a particular coverage of the document- empirically verified to be effective for its use- usable in existing methods, such as inspections• These goals defines a set of guidelines/characteristics for a process definition for reading techniques that can be studied experimentallyScenario-Based Reading DefinitionSo far, we have developed five families of reading techniques parameterized for use in different contexts and evaluated experimentally in those contextsThey include:perspective based reading:for detecting defects in requirements documents in Englishdefect based reading:for detecting defects in requirements documents in SCRscope based reading:for constructing designs from OO frameworksuse based reading:for detecting anomalies in user interface web screenshorizontal/vertical reading:for detecting defects in object oriented design in UMLEmpirically-Based DevelopmentExperimentation has been used to “validate” hypotheses, although most of the studies have been predominantly “exploratory”We are interested in using the empirical studies to help developand focus the technology, gain confidence that it is doing what it is supposed to be doing, and help evolve the definitionWe are especially interested in seeing that it works in the environment in which it is being appliedReading Technique Development: An Empirical Approach4. Execute5. Analyze6. Package1. Characterize:Model Important Aspects of a Document and How It Is Used3. Choose Process:Map Models toConcrete Procedure2. Set Goals:Determine Evaluation CriteriaMapping Models to a Reading TechniqueAbstractions of Information:A model of what information is important, and how it is best organized.Uses of Information:A model of the process by which the task is accomplished.Reading Technique:Tested practices for accomplishing a particular task• Need to characterize the “model of use”: how the information in a document is used for a particular task in a particular environment.Initial procedures for identifying information in this document that is relevantInitial procedures for using the information to accomplish the taskReading for Analysis: Scenario-Based Reading DefinitionSo far, two different scenario-based reading techniques have been defined for requirements documents:defect based readingperspective based readingDefect based reading focuses on different defect classes e.g., missing functionality and data type inconsistencies Existing definition for reading SCR style documentsPerspective-based reading focuses on different customer perspectives, e.g., reading from the perspective of the designer, tester, or end-userExisting definition for reading natural language requirements documents.Reading for Analysis: Perspective-Based Reading DefinitionDefect class Perspectiveanalysisgenerates questionsmodel generates scenarios Perspective based readingPerspective-Based ReadingAbstractions of Information:Design planTest planUser manualUses of Information:Check consistencyCheck completenessCheck correctness . . .Reading Technique:For detecting defects in requirementsAllow reviewers to usetheir usual proceduresCreate questions aimed atchecking each attributeAsk reviewers to create the appropriate abstraction, then answer a series of questions tailored to the abstractionReading for Analysis: Perspective-Based Reading DefinitionStakeholders for the requirements document include:Users, testers, designers, maintainers, …They all need to read the product in different waysHere we used three different perspectives:test-based readinguse-based readingdesigner-based readingTo identify faults, we might try to identify the following faultclasses:Omission, incorrect fact, ambiguity, inconsistencyReading for Analysis: Perspective-Based Reading DefinitionVarious customers of a product read it to find out if it satisfies their needsThe reader should find defects and assess the document from their particular point of view. We used three different perspectives:test-based readinguse-based readingdesigner-based readingExample: Test-based ReadingFor each requirement, make up a test or set of tests that will allow you to ensure that the implementation satisfies the requirement. Use your standard test approach and test criteria to make up the test suite. Reading for Analysis: Perspective-Based Reading Testing-Based Reading QuestionsFor each requirement, ask yourself the following questions:1. Do you have all the information necessary to identify the item being tested and to identify your test criteria? Can you make upreasonable test cases for each item based upon the criteria? 2. Is there another requirement for which you would generate a similar test case but would get a contradictory result?3. Can you be sure the test you generated will yield the correct value in the correct units?4. Are there other interpretations of this requirement that the implementor might make based upon the way the requirement is defined? Will this effect the test you made up?5. Does the requirement make sense from what you know about the application and from what is specified in the general description?Reading for Analysis: Blocked Subject Project StudyPerspective-Based Reading Study Goal:Analyze perspective-based reading,NASA’s current reading techniqueto evaluate and compare them with respect to their effect on fault detection effectiveness in the context of an inspection team from the viewpoint of quality assuranceEnvironment:NASA/CSC SEL EnvironmentRequirements documents: Two NASA Functional Specifications: ground support subsystemsTwo Structured Text Documents: ATM machine, Parking Garage All documents seeded with known defectsExperimental design:Metric: Percent of defects detectedBlocked subject-project: Partial factorial designReplicated twice: November 1994 and June 1995Subjects: 25 subjects in totalDesign of the PBR ExperimentPerspective-basedGroup 1 Group 2NASA doc. ANASA doc. ANASA doc. BNASA doc. BFirst dayTeachingPBRSecond dayPerspectives randomly and evenly assignedTraining in front of every test Same sizeGeneric partUsual approachATM networkATM networkParking garageParking garageReading for Analysis: Perspective-Based