University of Toronto Department of Computer Science Lecture 10 Validating Requirements Last Last Week Week Communicating CommunicatingRequirements Requirements Requirements RequirementsSpecifications Specifications Documentation DocumentationStandards Standards Requirements RequirementsTraceability Traceability This This Week Week Validating Validating Requirements Requirements Philosophical PhilosophicalIssues Issues Reviews Reviewsand andInspections Inspections Prototyping Prototyping Next Next Week Week Agreeing AgreeingRequirements Requirements Negotiation Negotiation Conflict ConflictResolution Resolution 2000 2003 Steve Easterbrook 1 University of Toronto Department of Computer Science Overview Two key problems for getting agreement 1 the problem of validation What is truth and what is knowable 2 the problem of negotiation How do you reconcile conflicting goals in a complex socio cognitive setting Validating Requirements Inspections and Reviews Prototyping Negotiating Requirements next week Conflict and Conflict Resolution Requirements Negotiation Techniques Argumentation approaches Knowledge based approaches Requirements Prioritization 2000 2003 Steve Easterbrook 2 1 University of Toronto Department of Computer Science The problem of validation logical positivist view there is an objective world that can be modeled by building a consistent body of knowledge grounded in empirical observation In RE assumes there is an objective problem that exists in the world Build a consistent model make sufficient empirical observations to check validity Use tools that test consistency and completeness of the model Use reviews prototyping etc to demonstrate the model is valid Popper s modification to logical positivism theories can t be proven correct they can only be refuted by finding exceptions In RE design your requirements models to be refutable Look for evidence that the model is wrong E g collect scenarios and check the model supports them post modernist view there is no privileged viewpoint all observation is value laden scientific investigation is culturally embedded E g Kuhn science moves through paradigms E g Toulmin scientific theories are judged with respect to a weltanschauung In RE validation is always subjective and contextualised Use stakeholder involvement so that they own the requirements models Use ethnographic techniques to understand the weltanschauungen 2000 2003 Steve Easterbrook 3 University of Toronto Department of Computer Science Reviews Inspections Walkthroughs Source Adapted from Blum 1992 pp369 373 Note these terms are not widely agreed formality informal from meetings over coffee to team get togethers formal scheduled meetings prepared participants defined agenda specific format documented output Management reviews E g preliminary design review PDR critical design review CDR Used to provide confidence that the design is sound Attended by management and sponsors customers Usually a dog and pony show Walkthroughs developer technique usually informal used by development teams to improve quality of product focus is on finding defects Fagan Inspections a process management tool always formal used to improve quality of the development process collect defect data to analyze the quality of the process written output is important major role in training junior staff and transferring expertise 2000 2003 Steve Easterbrook 4 2 University of Toronto Department of Computer Science Benefits of formal inspection Source Adapted from Blum 1992 pp369 373 Freedman and Weinberg 1990 Formal inspection works well for programming For applications programming more effective than testing most reviewed programs run correctly first time compare 10 50 attempts for test debug approach Data from large projects error reduction by a factor of 5 10 in some reported cases improvement in productivity 14 to 25 percentage of errors found by inspection 58 to 82 cost reduction of 50 80 for V V even including cost of inspection Effects on staff competence increased morale reduced turnover better estimation and scheduling more knowledge about defect profiles better management recognition of staff ability These benefits also apply to requirements inspections E g See study by Porter et al 2000 2003 Steve Easterbrook 5 University of Toronto Department of Computer Science Inspection Constraints Source Adapted from Blum 1992 pp369 373 Freedman and Weinberg 1990 Size enough people so that all the relevant expertise is available min 3 4 if author is present max 7 less if leader is inexperienced focus on small part of a design not the whole thing Duration product not ready find problems the author is already aware of concentration will flag if longer not too late Outputs all reviewers must agree on the result accept re work re inspect all findings should be documented summary report for management detailed list of issues 2000 2003 Steve Easterbrook Timing Examines a product once its author has finished it not too soon never more than 2 hours Scope product in use errors are now very costly to fix Purpose Remember the biggest gains come from fixing the process collect data to help you not to make the same errors next time 6 3 University of Toronto Department of Computer Science Inspection Guidelines Source Adapted from Freedman and Weinberg 1990 Prior to the review schedule Formal Reviews into the project planning train all reviewers ensure all attendees prepare in advance During the review review the product not its author keep comments constructive professional and task focussed stick to the agenda leader must prevent drift limit debate and rebuttal record issues for later discussion resolution identify problems but don t try to solve them take written notes After the review review the review process 2000 2003 Steve Easterbrook 7 University of Toronto Department of Computer Science Choosing Reviewers Source Adapted from Freedman and Weinberg 1990 Possibilities specialists in reviewing e g QA people people from the same team as the author people invited for specialist expertise people with an interest in the product visitors who have something to contribute people from other parts of the organization Exclude anyone responsible for reviewing the author i e line manager appraiser etc anyone with known personality clashes with other reviewers anyone who is not qualified to contribute all management anyone whose presence creates a conflict of interest 2000 2003 Steve Easterbrook 8 4 University of Toronto