This preview shows page 1-2-24-25 out of 25 pages.

Save
View full document
View full document
Premium Document
Do you want full access? Go Premium and unlock all 25 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 25 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 25 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 25 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 25 pages.
Access to all documents
Download any document
Ad free experience

Unformatted text preview:

CMSC433, Fall 2007 Programming Language Technologies and ParadigmsTestingBill PughSeptember 4th, 20072Readings for today• JUnit• How web servers work• Regular expressions– meaning of regular expressions (e.g., [0-9]+xx[0-9]+)– package java.util.regex.*– capturing groups• e.g., Pattern.compile("([a-z]+)([0-9]+)")• TCP Sockets and server socketsThings you should have completed• The survey• Getting Eclipse set up– plugins (course project manager, Clover)– linuxlab cvs repository• Checking out project 0• If you submitted project 0, perform a release test– several students submitted the project, haven’t performed a release test, and have errors3A student needs a note taker• A student needs someone to share notes for the course– Disability support services will pay you to share your notes. Real money ($). • See me if you are interested (today if at all possible)– Let me look at and copy your notes from today.4Binary Search• Jon Bentley says that he has asked hundreds of professional programmers to implement binary search– given them up to two hours to do so– only 10% implemented it correctly• Donald Knuth noted that while the first binary search algorithm was published in 1946– the first correct implementation wasn’t published until 19585Testing• Testing is almost never complete/exhaustive• But it can be obviously incomplete– like to avoid that• Want to run tests automatically• Want to identify regressions as soon as possible– a regression is when a test that had been working starts failing• Lots of small tests are good– better than one big monster test– when the monster test fails, you have lots of debugging to do6Test cases• Basic functionality– simplest possible test cases• Corner/Edge/Boundary cases– take a knob in the specification, and turn it all the way• Everything mentioned in the spec• Try to break the code– within the limits of the spec–7Code coverage• Tools exist to help you measure code coverage:– the portions of code executed• Any code that isn’t executed by any test case hasn’t been tested8Several kinds of code coverage• Statement coverage– is a statement executed?• Branch coverage– is each branch in the program explored?• Boolean coverage– does each boolean expression evaluate to both true and false?• Path coverage– Is each (feasible) path through a method covered?• not all paths are feasible• Data coverage9Why isn’t statement coverage enough?int f(Object x) { int result = 5; if (x != null) result += x.hashCode(); return result*x.hashCode(); }10Boolean coverage• How many test cases should we have for:if (a > 0 && b > 0) foo();• How about – a = 1, b = 1– a = -1, b = -1• Should we also test the case where b > 0 evaluates to false?– a = 1, b = -111Path coverage• How many test cases should we have for:if (a > 0) foo();if (b > 0) bar();• We need 4 cases to cover all paths12a > 0foo()b > 0bar()Infeasible paths• How many test cases should we have for:if (a > 0) foo();if (a > 5) bar();• One of the paths isn’t feasible13a > 0foo()a > 5bar()Code coverage is nice• Can be automatically computed• Can spot coverage regressions– did you commit a change that reduced code coverage?• perhaps you introduced new code without a test case• perhaps you changed how execution flows so that preexisting code is no longer executed• Managers can measure it– Good managers will understand it and use it well– Some managers insist on high or perfect code coverage14Code coverage isn’t enough• Test cases that obtain code coverage but don’t check the results sort of miss the point– still useful• A lot of times you need data coverage– did you try duplicated elements?– did you try negative values in the array?15Jester• Mutation testing– Modifies your code– checks to see which code changes don’t result in test cases failing– may indicate lack of coverage• http://jester.sourceforge.net/16Writing tests isn’t easy• Some experience/evidence to suggest that to really test your code, you will need to write more lines of test code than lines of code being tested• We found over the years that the ratio of code to test code very rarely changes: Typically for every line of Java, it takes 3 to 5 lines of JUnit to achieve 80-90 percent code coverage. If you have 250,000 lines of Java code, you probably need about one million lines of JUnit. Writing tests is hard for anybody, and writing tests for legacy code, especially that many tests, is no fun. You are looking at many engineering years of effort.– Alberto Savoia, Agitar software– http://www.artima.com/forums/flat.jsp?forum=276&thread=21340217http://www.junitfactory.com/• Automated generation of JUnit characterization tests– doesn’t know what your code is supposed to do• Tries to generate test cases that achieve full coverage (statement or branch?)• Asserts that the code does whatever it does at generation time– may not be correct– check test cases to see if the asserted behavior is correct– use tests to check for changes in behavior• for good or bad18Writing tests isn’t easy• You may need to rewrite/refactor your code to make it easy to test your code– pull out logic so that it is easier to test– create mock objects• dummy objects that the code under test will interoperate with– allow for failure injection• how can you test how your code behaves when the network fails unless you can force the network to fail on demand19An amusing story, and a few lessons• A standard part of binary search implementations:int mid = (low + high) / 2;• What is wrong with this code?20This code is bad • This had been used in Sun’s JDK binary search implementation for many years• What could go wrong?• What is the array is very large?– more than 2^30 elements• What if (low+high) overflows 32-bit signed arithmetic?– then you get a negative number– which remains negative when you divide by 221Several ways to fix it• Easiest fix: use unsigned right shiftint mid = (low + high) >>> 1;• Shift result one bit to the right, filling in with a zero in the high order bit– result is always nonnegative– pretty sure it gives us what we want• How do we test it?22Testing it• This won’t be tested for project 0, but...• We could create a byte array containing more than 2^30 elements– might even be able to test that on my laptop– harder with


View Full Document

UMD CMSC 433 - Testing

Documents in this Course
Trace 1

Trace 1

62 pages

Reflection

Reflection

137 pages

Paradigms

Paradigms

10 pages

Testing

Testing

17 pages

Java RMI

Java RMI

17 pages

Java RMI

Java RMI

17 pages

Java RMI

Java RMI

17 pages

Trace 1

Trace 1

46 pages

Jini

Jini

4 pages

Final

Final

15 pages

Java RMI

Java RMI

13 pages

Testing

Testing

16 pages

Load more
Download Testing
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Testing and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Testing 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?