TestingGedankenSlide 3Goals of this LectureProgram VerificationProgram TestingExternal vs. Internal TestingExternal TestingBoundary TestingBoundary Testing ExampleBoundary Testing Example (cont.)Slide 12Slide 13Ambiguity in SpecificationMorals of This Little StoryStatement TestingStatement Testing ExamplePath TestingPath Testing ExampleStress TestingStress Testing Example 1Stress Testing Example 2The assert MacroUses of assertDisabling assertsDisabling asserts (cont.)Internal TestingTesting InvariantsTesting Invariants (cont.)Verifying Conservation PropertiesChecking Return ValuesChecking Return Values (cont.)Changing Code TemporarilyLeaving Testing Code IntactGeneral Testing StrategiesTesting IncrementallyTesting Incrementally (cont.)Comparing ImplementationsAutomationBug-Driven TestingFault InjectionWho Tests WhatSummarySummary (cont.)1TestingThe material for this lecture is drawn, in part, fromThe Practice of Programming (Kernighan & Pike) Chapter 62Gedanken•Given the realities of finite resources•What would help you the most in class?•Changes to the precepts?•Changes to the lectures?•Changes to the anything else?•Observations: end-of-semester evaluations useful for the next time. Find what’s useful now•No signatures needed, etc., etc.•Can also anonymously e-mail me if longer response desired3“On two occasions I have been asked [by members of Parliament!], ‘Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?’ I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.” ‒ Charles Babbage “Program testing can be quite effective for showing the presence of bugs, but is hopelessly inadequate for showing their absence.” ‒ Edsger Dijkstra“Beware of bugs in the above code; I have only proved it correct, not tried it.” Donald Knuth ‒4Goals of this Lecture•Help you learn about:•Internal testing•External testing•General testing strategies•Why?•It can be very difficult to determine if a large program works properly•When developing a large program, a power programmer expends at least as much effort writing test code as he/she expends writing the program itself•A power programmer is comfortable with a wide variety of program testing techniques and strategies5Program Verification•Ideally: Prove that your program is correct•Can you prove properties of the program?•Can you prove that it even terminates?!!!ProgramCheckerprogram.cRight/WrongSpecification?6Program Testing•Pragmatically: Convince yourself that your program probably worksTestingStrategyprogram.cProbablyRight/WrongSpecification7External vs. Internal Testing•Types of testing•External testing•Designing data to test your program•Internal testing•Designing your program to test itself8External Testing•External testing: Designing data to test your program•External testing taxonomy(1) Boundary testing(2) Statement testing(3) Path testing(4) Stress testing•Let’s consider one at a time…9Boundary Testing(1) Boundary testing•“A testing technique using input values at, just below, and just above, the defined limits of an input domain; and with input values causing outputs to be at, just below, and just above, the defined limits of an output domain.” ‒ Glossary of Computerized System and Software Development Terminology•Alias corner case testing•Almost all bugs occur at boundary conditions •If program works for boundary conditions, it probably works for all others10Boundary Testing Example•Truncate the last letter of a word•Example•Input "foot" output "foo"•But, what if the input is the empty line (i.e., just a '\n')?•Or EOF?•Or a very long string?int i;char s[MAXLINE];for (i=0; (s[i]=getchar()) != '\n'; i++) ;s[--i] = '\0';printf("String: %s\n", s);11Boundary Testing Example (cont.)•Code to get line from stdin and put in character array•Boundary conditions•Input starts with '\n' (empty line)•End of file before '\n'•End of file immediately (empty file)•Line exactly MAXLINE-1 characters long•Line exactly MAXLINE characters long•Line more than MAXLINE characters longint i;char s[MAXLINE];for (i=0; (s[i]=getchar()) != '\n' && i < MAXLINE-1; i++) ;s[i] = '\0';printf("String: |%s|\n", s);12Boundary Testing Example (cont.)•Rewrite the code•Another boundary condition: EOF•What are other boundary conditions?•Nearly full•Exactly full•Over fullint i;char s[MAXLINE];for (i=0; i<MAXLINE-1; i++) if ((s[i] = getchar()) == '\n') break;s[i] = '\0';for (i=0; i<MAXLINE-1; i++) if ((s[i] = getchar()) == '\n' || s[i] == EOF) break;s[i] = '\0';This is wrong.Why?13Boundary Testing Example (cont.)•Rewrite yet againOutput:•There’s still a problem...Input:Fourscore and sevenyearsFourØscore anØ sevenØyearsØWhere’s the ‘d’?for (i=0; ; i++) { int c = getchar(); if (c==EOF || c=='\n' || i==MAXLINE-1) { s[i] = '\0'; break; } else s[i] = c;}14Ambiguity in Specification•If line is too long, what should happen?•Keep first MAXLINE characters, discard the rest?•Keep first MAXLINE-1 characters + '\0' char, discard the rest?•Keep first MAXLINE-1 characters + '\0' char, save the rest for the next call to the input function?•Probably, the specification didn’t even say what to do if MAXLINE is exceeded•Probably the person specifying it would prefer that unlimited-length lines be handled without any special cases at all•Moral: testing has uncovered a design problem, maybe even a specification problem!•Define what to do•Truncate long lines?•Save the rest of the text to be read as the next line?15Morals of This Little Story•Complicated, messy boundary cases are often symptomatic of bad design or bad specification•Clean up the specification if you can•If you can’t fix the specification, then fix the code16Statement Testing(2) Statement testing•“Testing to satisfy the criterion that each statement in a program be executed at least once during program testing.” ‒ Glossary of Computerized System and Software Development Terminology17Statement Testing Example•Example pseudocode:•Requires two data sets; example:•condition1 is true and condition2 is true•Executes statement1 and statement3•condition1 is false and condition2 is false•Executes statement2 and statement4if (condition1) statement1;else statement2;…if (condition2) statement3;else
View Full Document
Unlocking...