CMSC 433 – Programming Language Technologies and ParadigmsFall 2002Developing, Testing, DebuggingSome slides adapted from FSE’98 Tutorial by Michal Young and Mauro Pezze’Tools, Testing, Debugging•Goal: write reliable, correct software–And do it without too much pain!•The development process–Analyze requirements (what am I trying to do?)–Write code (how am I doing it?)–Test code (does my code work?)–Debug code (nope—what’s wrong?)–then iterate! (has what I’ve written met the requirements?)Holistic Development: IDE’s•Interactive Development Environments–Typically simplify writing of code (editor + compiler hook)–May help with debugging (syntax checking, debugger interface)–May help with testing (interface to test tool)–May help with search, transformation, etc.•Many available–Eclipse JDT–Dr. Java–Visual Studio– …Dr. Java•Editing–Syntax coloring– Auto-indent–Brace matching• Testing–Integrates with Junittesting framework–Interaction panel allows interactive method invocations•Debugging–Integrates with Java debugger–Interactions panel also usefulTestingHow do I know my program works?•Identifyproperties of interest•Formally provethey are upheld, or•Test themby running with various inputs•But which properties? How do I do it? What if my program changes? When am I done? Ahhh!!!•It gets worse …You can’t always get what you want •Correctness properties are undecidable–the halting problem can be embedded in almost every property of interestDecisionProcedurePropertyProgramPass/FaileverGetting what you need ...•We must make the problem of verification “easier” by permitting some kind of inaccuracyPerfectverificationOptimistic inaccuracy(testing)Pessimistic inaccuracy(analysis, proofs)“Easier” propertiesEasier Properties -Example:Unmatched Semaphore Operationssynchronized(S) {...... }Static checking for match is necessarily inaccurate ... ... so Java prescribes a more restrictive, but statically checkable construct. if ( .... ) {...lock(S); }...if ( ... ) {...unlock(S); }Testing•Executing the program on a sample of input data•Dynamic technique: programs must be executed•Optimistic inaccuracy–the program runs on a (very small) subsetof input data–the behavior of the program on everyinput is assumed to be consistent with the examined behaviorsGoals of Testing•Find faults (“Debug” Testing):–a test is successful iffthe program fails1•Provide confidence (Acceptance Testing)–of reliability –of (probable) correctness–of detection (therefore absence) of particular faults1.Goodeneogh, Gerhart, “Toward a Theory of Test Data Selection”, IEEE Transactions on Software Engineering, Jan. 85Testing Activities•Test case execution is only a part of the process•Must also consider–Test case generation–Test result evaluation• Planning is essential–To achieve early and continuous visibility–To choose appropriate techniques at each stage–To build a testable product–To coordinate complementary analysis and testingThe Test Case Generation Problem•What tests will show that my program works?–Must consider “operational scenarios”–What is legitimate input?–What is the correct action or output?•How can I make sure that all of the important behaviors of my program have been tested?Granularity of Tests•Whole program–Test case inputs to whole program, and outputs examined•Piece-meal–Individual components of a program are tested• Methods•Classes/packages•Processes of a distributed systemStyles of Tests•Functional (black box)–based on specifications (“external behavior”)•Structural (white box)– based on code•Fault based–based on classes of faultsBlack Box Testing•Pick a subcomponent of the program–Internals of component not considered•Give it inputs•Compare against correct outputsinputsoutputssubcomponentbeing testedIs it correct?oracleWhite Box Testing•Pick a subcomponent of the program•Give it inputs–Inputs determined based on component code•Compare against correct outputsinputsoutputssubcomponentbeing testedIs it correct?oracleWhite vs. Black box•Black box–depends on spec–scales up•different techniques at different granularity levels–it cannot reveal code coverage problems•same specification implemented with different modules•White box–depends on control or data flow coverage–does not scale up•mostly applicable at unit and integration testing level–cannot reveal missing path errors•part of the specification that is not implementedThe Testing Environment•Want to create a scaffold for executing tests–Code infrastructure that allows tests to be run easily and the results checked for correctness•Many benefits–Can automate testing process– Useful for regression testing•But, can take some time to implementTesting Environment Components•A userto generate input for tested component• An oraclefor verifying the results are correct•These two may be combined into a single systemUnit Testing withJunit•Testing environment for writing black-box tests–Write special TestCaseclasses to test other classes–Several ways to use/set up test cases•Can be downloaded from –http://www.junit.org•Simple version included in Dr. Java–Implements only a subset of JunitfunctionalityJunitPhilosophy• Iterative, incremental process–Write small black-box test cases (as needed)–Test-as-you-go•i.e., after changes, when new method added, when bug identified•Junittest cases must be completely automated–No human judgment–Easy to run many of them at the same time•Goal: lots of bang for the buck–Even simple tests can find many bugs quicklyJunit Components•Test cases (class TestCase)–Individual tests–Can reuse test case setup (optional)•Test suites (class TestSuite)–Test case container•Test runner (various classes)–Executes test suites and presents resultsEach test has three 3 parts•Code that creates test objects–Create a subclass of junit.framework.TestCase•Code that executes the test–Override the method runTest() (which executes the test)•Code that verifies the result–e.g., use junit.framework.assertTrue() to check results (throws exception is test fails)TestCase Example with LogRecordpublic class LogRecordTestextends TestCase {protected String event1 = “test”;public void testEquals1() {LogRecordtmp1 = new LogRecord(event1+”1”); LogRecordtmp2 = new
View Full Document
Unlocking...