1University of TorontoDepartment of Computer Science© 2008 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 1Lecture 15:Verification and ValidationRefresher: definitions of V&VV&V strategiesModeling and PrototypingInspectionFormal Analysis(Testing)Independent V&VQuality AssuranceUniversity of TorontoDepartment of Computer Science© 2008 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 2Refresher: V&VValidation:“Are we building the rightsystem?”Does our problem statementaccurately capture the realproblem?Did we account for the needs ofall the stakeholders?Verification:“Are we building the systemright?”Does our design meet the spec?Does our implementation meetthe spec?Does the delivered system dowhat we said it would do?Are our requirements modelsconsistent with one another?ProblemStatementImplementationStatementSystemValidationVerificationProblemSituation2University of TorontoDepartment of Computer Science© 2008 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 3VerificationTraditional approaches to (code) verificationModel-based verificationexperiment withthe program(testing)reason aboutthe program(static verification)inspect theprogram(reviews)do the use casessatisfy the requirements?(goal analysis)does the codecorrespond to the model?(consistency checking)does the class modelsatisfy the use cases?(robustness analysis)University of TorontoDepartment of Computer Science© 2008 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 4Understanding ValidationPrior Knowledge(e.g. customer feedback)Observe(what is wrong withthe current system?)Model(describe/explain theobserved problems)Design(invent a better system)Intervene(replace the old system)Note similarity withprocess of scientificinvestigation:Requirements models aretheories about the world;Designs are tests of thosetheoriesInitial hypothesesLook for anomalies - what can’tthe current theory explain?Create/refinea better theoryDesign experiments totest the new theoryCarry out theexperiments(manipulatethe variables)3University of TorontoDepartment of Computer Science© 2008 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 5Validation techniquesPrior Knowledge(e.g. customer feedback)Observe(what is wrong withthe current system?)Model(describe/explain theobserved problems)Design(invent a better system)Intervene(replace the old system)Build aPrototypeGet usersto try it(what is wrong withthe prototype?)Analyzethe modelrun a modelchecking tool(what is wrong withthe model?)Inspectthe modelUniversity of TorontoDepartment of Computer Science© 2008 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 6Choice of TechniquesVerification Validationcode inspectionstatic analysistestingprototypingusability testgoal analysisunit testacceptance testintegration testautomatedtesting model/spec inspectionmodel checkingproofs of correctnessstyle checkersrobustnessanalysisconsistencycheckingbeta testsystem testregression testmodeling??4University of TorontoDepartment of Computer Science© 2008 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 7PrototypingPresentation Prototypesexplain, demonstrate and inform – then throw awaye.g. used for proof of concept; explaining design features; etc.Exploratory Prototypesused to determine problems, elicit needs, clarify goals, compare design optionsinformal, unstructured and thrown away.Breadboards or Experimental Prototypesexplore technical feasibility; test suitability of a technologyTypically no user/customer involvementEvolutionary(e.g. “operational prototypes”, “pilot systems”):development seen as continuous process of adapting the system“prototype” is an early deliverable, to be continually improved.University of TorontoDepartment of Computer Science© 2008 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 8Usability TestingReal users try out the system (or prototype)Choose representative tasksChoose representative usersObserve what problems they encounterHow many users?3-5 users gives best return on investment5University of TorontoDepartment of Computer Science© 2008 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 9Formal Model AnalysisVerification“Is the model well-formed?”Are the parts of the model consistent with one another?Validation:Animation of the model on small examplesFormal challenges:“if the model is correct then the following property should hold...”‘What if’ questions:reasoning about the consequences of particular requirements;reasoning about the effect of possible changes“will the system ever do the following...”State explorationE.g. use model checking to find traces that satisfy some propertyUniversity of TorontoDepartment of Computer Science© 2008 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 10UML Consistency CheckingUse Case DiagramsDoes each use case have a user?Does each user have at least one use case?Is each use case documented?Using sequence diagrams or equivalentClass DiagramsDoes the class diagram capture all the classes mentioned in other diagrams?Does every class have methods to get/set its attributes?Sequence DiagramsIs each class in the class diagram?Can each message be sent?Is there an association connecting sender and receiver classes on the class diagram?Is there a method call in the sending class for each sent message?Is there a method call in the receiving class for each received message?6University of TorontoDepartment of Computer Science© 2008 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 11Model CheckersAutomatically check properties (expressed in Temporal Logic)temporal logic adds modal operators to FOPL:e.g. p - p is true now and always (in the future)e.g.
View Full Document
Unlocking...