1!University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 1 Lecture 17:!Testing Strategies"Structural Coverage Strategies (White box testing):"Statement Coverage"Branch Coverage"Condition Coverage"Data Path Coverage"Function Coverage Strategies (Black box testing):"Use Cases as Test Cases"Testing with good and bad data"Stress Testing"Quick Test"Interference Testing"A radical alternative: Exploratory Testing""University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 2 Developer Testing"Write the test cases first"minimize the time to defect discovery"forces you to think carefully about the requirements first"exposes requirements problems early"supports a “daily smoke test”"But: Limitations of Developer Testing"Emphasis on clean tests (vs. dirty tests)"immature organisations have "1 dirty : 5 clean"mature organisations have "5 dirty : 1 clean"Developers overestimate test coverage"Developers tend to focus on statement coverage rather than …"Summary:"Test-case first strategy is extremely valuable"Test-case first strategy is not enough"2!University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 3 Structured Basis Testing"The minimal set of tests to cover every branch"How many tests?"start with 1 for the straight path"add 1 for each of these keywords: if, while, repeat, for, and, or"add 1 for each branch of a case statement"Example"int midval (int x, y, z) { /* effects: returns median value of the three inputs */ if (x > y) { if (x > z) return x else return z } else { if (y > z) return y else return z } } Count 1 + 3 ʻifʼs = 4 test cases"Now choose the cases to exercise the 4 paths:"e.g. "x=3, y=2, z=1"" "x=3, y=2, z=4"" "x=2, y=3, z=2"" "x=2, y=3, z=4"Source: Adapted from McConnell 2004, p506-508 University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 4 Complex Conditions"If ((a) & (b) | (c)) then… true false Branch Coverage:!If ((a) & (b) | (c)) then… true false true false true false Condition Coverage:!If ((a) & (b) | (c)) then… true false true false But can you show that!each part has an independent!effect on the outcome?!Source: Adapted from Christopher Ackermann’s slides3!University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 5 MC/DC Coverage"Show that each basic condition can affect the result"Source: Adapted from Christopher Ackermann’s slides If ((a) & (b) | (c)) then… (1) Compute truth table for the condition"(2) In each row, identify any case where flipping one variable changes the result."(3) Choose a minimal set:""Eg. {2, 3, 4, 6}""or {2, 3, 4, 7}"(4) Write a test case for each element of the set"Number" ABC" Result" A" B" C"1! TTT! T! 5!2! TTF! T! 6! 4!3! TFT! T! 7! 4!4! TFF! F! 2! 3!5! FTT! F! 1!6! FTF! F! 2!7! FFT! F! 3!8! FFF! F!University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 6 MC/DC versus Branch Coverage"Compiler can translate conditions in the object code:"""""Test sets for Condition/Decision coverage:"{1, 8} or {2, 7} or {3, 6}"Covers all paths in the source code, but not all paths in the object code"Test sets for Modified Condition/Decision Coverage"{2, 3, 4, 6} or {2, 3, 4, 7}"Covers all paths in the object code"Source: Adapted from Christopher Ackermann’s slides4!University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 7 About MC/DC"Advantages:"Linear growth in the number of conditions"Ensures coverage of the object code"Discovers dead code (operands that have no effect)"Mandated by the US Federal Aviation Administration"In avionics, complex boolean expressions are common"Has been shown to uncover important errors not detected by other test approaches"Itʼs expensive"E.g. Boeing 777 (first fly-by-wire commercial aircraft)"approx 4 million lines of code, 2.5 million newly developed"approx 70% of this is Ada (the rest is C or assembler)"Total cost of aircraft development: $5.5 billion"Cost of testing to MC/DC criteria: approx. $1.5 billion"University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 8 Dataflow testing"Things that happen to data:"Defined - data is initialized but not yet used"Used - data is used in a computation"Killed - space is released"Entered - working copy created on entry to a method"Exited - working copy removed on exit from a method"Normal life:"Defined once, Used a number of times, then Killed"Potential Defects:"D-D: variable is defined twice"D-Ex, D-K: variable defined but not used"En-K: destroying a local variable that wasnʼt defined?"En-U: for local variable, used before itʼs initialized"K-K: unnecessary killing - can hang the machine?"K-U: using data after it has been destroyed"U-D: redefining a variable after is has been used"Source: Adapted from McConnell 2004, p506-5085!University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 9 Testing all D-U paths"The minimal set of tests to cover every D-U path"How many tests?"1 test for each path from each definition to each use of the variable"Example"if (Cond1) { x = a; } else { x = b; } if (Cond2) { y = x + 1 } else { y = x - 1; } Structured Basis Testing:"2 test cases is sufficient"Case 1: Cond1=true, Cond2=true"Case 2: Cond1=false, Cond2=false"All DU testing:"Need 4 test cases"Source: Adapted from McConnell 2004, p506-508 D:!D:!U:!U:!University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This
View Full Document
Unlocking...