1!University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 1 Lecture 18:!Automated Testing"""Automated testing"JUnit and family"Testing GUI-based software"Testing Object-Oriented Systems"When to stop testing""University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 22!University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 3 Automated Testing"Where possible, automate your testing:"tests can be repeated whenever the code is modified (“regression testing”)"takes the tedium out of extensive testing"makes more extensive testing possible"Will need:"test drivers - automate the process of running a test set"sets up the environment"makes a series of calls to the Unit-Under-Test (UUT)"saves results and checks they were right"generates a summary for the developers"May need:"test stubs - simulate part of the program called by the unit-under-test"checks whether the UUT set up the environment correctly"checks whether the UUT passed sensible input parameters to the stub"passes back some return values to the UUT (according to the test case)"(stubs could be interactive - ask the user to supply return values)"Source: Adapted from Liskov & Guttag, 2000, pp239-242 University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 4 Automated Testing Strategy"Setup!Exercise!Verify!Teardown!UUT!Unit!Under!Test!DOC!Depended!On !Component!Initialize!Exercise !(with return value)!Get State!Get Something!(with return value)!Do something!(no return value)!Direct control points!Indirect control point!Direct observation points!Indirect observation points!TestCase!Fixture!Test Double!Source: Adapted from Meszaros 2007, p663!University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 5 Test Order?"Inside!Out!Outside!In!Source: Adapted from Meszaros 2007, p35 UUT!UUT!UUT!UUT!UUT!UUT!University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 6 How JUnit works"Source: Adapted from Meszaros 2007, p77 UUT!4!University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 7 How JUnit works"Source: Adapted from Meszaros 2007, p77 UUT!University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 8 Assertion methods in JUnit"Single-Outcome Assertions"fail;"Stated Outcome Assertions"assertNotNull(anObjectReference);"assertTrue(booleanExpression)"Expected Exception Assertions"assert_raises(expectedError) {codeToExecute };"Equality Assertions"assertEqual(expected, actual);"Fuzzy Equality Assertions"assertEqual(expected, actual, tolerance);"Source: Adapted from Meszaros 2007, p3655!University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 9 Principles of Automated Testing"Write the Test Cases First"Design for Testability"Use the Front Door First"test via public interface"avoid creating back door manipulation"Communicate Intent"Tests as Documentation!"Make it clear what each test does"Donʼt Modify the UUT"avoid test doubles"avoid test-specific subclasses"(unless absolutely necessary)"Keep tests Independent"Use fresh fixtures"Avoid shared fixtures"Isolate the UUT"Minimize Test Overlap"Check One Condition Per Test"Test Concerns Separately"Minimize Untestable code"e.g. GUI components"e.g. multi-threaded code"etc"Keep test logic out of production code"No test hooks!""Source: Adapted from Meszaros 2007, p39-48 University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 10 Testing interactive software"1) Start the application (e.g. UMLet)!2) Click on! File -> Open!3) select test2.uxf!4) click Open!6!University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 11 Automating the testing"Challenges for automated testing:"Synchronization - How do we know a window popped open that we can click in?"Abstraction - How do we know itʼs the right window?"Portability - What happens on a display with different resolution / size, etc"Units!Functionality!Presentation!Automated!tests!Manual!tests!Source: Adapted from Zeller 2006, p57 University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 12 Testing the Presentation Layer"Script the mouse and keyboard events"script can be recorded (e.g. “send_xevents @400,100”)"script is write-only and fragile"Script at the application function level"E.g. Applescript: tell application “UMLet”to activate Robust against size and position changes"Fragile against widget renamings, layout changes, etc."Write an API for your application…"Allow an automated test to create windows, interact with widgets, etc." Source: Adapted from Zeller 2006, chapter 37!University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 13 Dealing with Circular Dependencies"Core!+print_to_file()!UserPresentation!+confirm_loss()!void print_to_file(string filename) !{! if (path_exists(filename)) {! // FILENAME exists; ask user to confirm overwrite! bool confirmed = confirm_loss(filename);! if (!confirmed)!
View Full Document
Unlocking...