1!University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 1 Lecture 15:!Verification and Validation"Refresher: definitions of V&V"V&V strategies"Modeling and Prototyping"Inspection"Formal Analysis"(Testing)"Independent V&V"Quality Assurance"University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 2 Refresher: V&V"Validation:"“Are we building the right system?”"Does our problem statement accurately capture the real problem?"Did we account for the needs of all the stakeholders?"Verification:"“Are we building the system right?”"Does our design meet the spec?"Does our implementation meet the spec?"Does the delivered system do what we said it would do?"Are our requirements models consistent with one another?"Problem Statement Implementation Statement System Validation Verification Problem Situation2!University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 3 Verification"Traditional approaches to (code) verification"""""Model-based verification"experiment with!the program!(testing)!reason about!the program!(static verification)!inspect the!program!(reviews)!do the use cases!satisfy the requirements?!(goal analysis)!does the code!correspond to the model?!(consistency checking)!does the class model satisfy the use cases?!(robustness analysis)!University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 4 Understanding Validation"Prior Knowledge (e.g. customer feedback) Observe (what is wrong with the current system?) Model (describe/explain the observed problems) Design (invent a better system) Intervene (replace the old system) Note similarity with process of scientific investigation: Requirements models are theories about the world; Designs are tests of those theories Initial hypotheses Look for anomalies - what canʼt the current theory explain? Create/refine a better theory Design experiments to test the new theory Carry out the experiments (manipulate the variables)3!University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 5 Validation techniques"Prior Knowledge (e.g. customer feedback) Observe (what is wrong with the current system?) Model (describe/explain the observed problems) Design (invent a better system) Intervene (replace the old system) Build a Prototype Get users to try it (what is wrong with the prototype?) Analyze the model run a model checking tool (what is wrong with the model?) Inspect the model University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 6 Choice of Techniques"Verification! Validation!code "inspection!static analysis!testing!prototyping!usability !test!goal analysis!unit test!acceptance" test!integration "test!automated!testing! model/spec "inspection!model "checking!proofs of "correctness!style "checkers!robustness!analysis!consistency!checking!beta test!system test!regression! test!modeling!?!?!4!University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 7 Prototyping"Presentation Prototypes"Explain, demonstrate and inform – then throw away"e.g. used for proof of concept; explaining design features; etc."Exploratory Prototypes"Used to determine problems, elicit needs, clarify goals, compare design options"Informal, unstructured and thrown away."Breadboards or Experimental Prototypes"Explore technical feasibility, or test suitability of a technology, then thrown away"Typically no user/customer involvement"Evolutionary "(a.k.a. “operational prototypes”, “pilot systems”):"Development seen as continuous process of adapting the system"“prototype” is an early deliverable, to be continually improved."University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 8 Usability Testing"Real users try out the system (or prototype)"Choose representative tasks"Choose representative users"Observe what problems they encounter"How many users?"3-5 users gives best return on investment"5!University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 9 Model Analysis"Verification"“Is the model well-formed?”"Are the parts of the model consistent with one another?"Validation:"ʻWhat ifʼ questions:"reasoning about the consequences of particular requirements;"reasoning about the effect of possible changes"“will the system ever do the following...” "Formal challenges:"“if the model is correct then the following property should hold...”"Animation of the model on small examples"State exploration "E.g. use model checking to find traces that satisfy some property"University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 10 UML Consistency Checking"Use Case Diagrams"Does each use case have a user?"Does each user have at least one use case?"Is each use case documented?"Using sequence diagrams or equivalent"Class Diagrams"Does the class diagram capture all the classes mentioned in other diagrams?"Does every class have methods to get/set its attributes?"Sequence Diagrams"Is each class in the class diagram?"Can each message be sent?"Is there an association connecting sender and receiver classes on the class diagram?"Is there a method call in the sending class for each sent message?"Is there a method call in the receiving class for each received message?"6!University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This
View Full Document
Unlocking...