11TestingThe material for this lecture is drawn, in part, fromThe Practice of Programming (Kernighan & Pike) Chapter 6Professor Jennifer Rexfordhttp://www.cs.princeton.edu/~jrex2Bugs, Bugs Everywhere“On two occasions I have been asked [by members of Parliament!], ‘Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?’ I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.”‒ Charles Babbage“Program testing can be quite effective for showing the presence of bugs, but is hopelessly inadequate for showing their absence.”‒ Edsger Dijkstra“Beware of bugs in the above code; I have only proved it correct,not tried it.”‒ Donald Knuth23Goals of this Lecture• Help you learn about:• Internal testing• External testing• General testing strategies•Why?• Hard to know if a large program works properly• When developing a large program, a power programmer expends at leastas much effort writing test code as he/she expends writing the program itself• A power programmer is comfortable with a wide variety of program testing techniques and strategies4Program Verification• Ideally: Prove that your program is correct• Can you prove properties of the program?• Can you prove that it even terminates?!!!ProgramCheckerprogram.cRight/WrongSpecification?35Program Testing• Pragmatically: Convince yourself that your program probably worksTestingStrategyprogram.cProbablyRight/WrongSpecification6External vs. Internal Testing• Types of testing•Externaltesting• Designing data to test your program• Internal testing• Designing your program to test itself47External Testing• External testing: Designing data to test your program• External testing taxonomy(1) Boundary testing(2) Statement testing(3) Path testing(4) Stress testing• Let’s consider one at a time…8Boundary Testing(1) Boundary testing• “A testing technique using input values at, just below, and just above, the defined limits of an input domain; and with input values causing outputs to be at, just below, and just above, the defined limits of an output domain.”‒Glossary of Computerized System and Software Development Terminology•Alias corner case testing• Almost all bugs occur at boundary conditions • If program works for boundary conditions, it probably works for all others59Boundary Testing Example• Code to get line from stdin and put in character array• Boundary conditions• Input starts with '\n' (empty line)• Prints empty string (“\0”), so output is “||”• End of file before '\n‘• Keeps calling getchar() and storing ӱ in s[i]• End of file immediately (empty file)• Keeps calling getchar() and storing ӱ in s[i]int i;char s[MAXLINE];for (i=0; (s[i]=getchar()) != '\n' && i < MAXLINE-1; i++);s[i] = '\0';printf("String: |%s|\n", s);10Boundary Testing Example (cont.)• Code to get line from stdin and put in character array• Boundary conditions• Line exactly MAXLINE-1 characters long• Output is correct, with ‘\0’ in s[MAXLINE-1]• Line exactly MAXLINE characters long• Last character on the line is overwritten, and newline never read• Line more than MAXLINE characters long• Some characters, plus newline, not read and remain on stdinint i;char s[MAXLINE];for (i=0; (s[i]=getchar()) != '\n' && i < MAXLINE-1; i++);s[i] = '\0';printf("String: |%s|\n", s);611Boundary Testing Example (cont.)• Rewrite the code• Another boundary condition: EOF• What are other boundary conditions?•Nearly full• Exactly full•Over fullint i;char s[MAXLINE];for (i=0; i<MAXLINE-1; i++)if ((s[i] = getchar()) == '\n')break;s[i] = '\0';for (i=0; i<MAXLINE-1; i++)if ((s[i] = getchar()) == '\n' || s[i] == EOF)break;s[i] = '\0';This is wrong.Why?12Boundary Testing Example (cont.)• Rewrite yet againOutput:• There’s still a problem...Input:Fourscore and sevenyearsFourØscore anØsevenØyearsØWhere’s the ‘d’?for (i=0; ; i++) {int c = getchar();if (c==EOF || c=='\n' || i==MAXLINE-1) {s[i] = '\0'; break;}else s[i] = c;}713Ambiguity in Specification• If line is too long, what should happen?• Keep first MAXLINE characters, discard the rest?• Keep first MAXLINE-1 characters + '\0' char, discard the rest?• Keep first MAXLINE-1 characters + '\0' char, save the rest for the next call to the input function?• Probably, the specification didn’t even say what to do if MAXLINE is exceeded• Probably the person specifying it would prefer that unlimited-length lines be handled without any special cases at all• Moral: testing has uncovered a design problem, maybe even a specification problem!• Define what to do• Truncate long lines?• Save the rest of the text to be read as the next line?14Morals of This Little Story• Complicated, messy boundary cases are often symptomatic of bad design or bad specification• Clean up the specification if you can• If you can’t fix the specification, then fix the code815Statement Testing(2) Statement testing• “Testing to satisfy the criterion that each statement in a program be executed at least once during program testing.”‒Glossary of Computerized System and Software Development Terminology16Statement Testing Example• Example pseudocode:• Requires two data sets; example:• condition1 is true and condition2 is true• Executes statement1 and statement3• condition1 is false and condition2 is false• Executes statement2 and statement4if (condition1)statement1;elsestatement2;…if (condition2)statement3;elsestatement4;…Statement testing:Should make sure both “if”statements and all 4 nested statements are executed917Path Testing(3) Path testing• “Testing to satisfy coverage criteria that each logical path through the program be tested. Often paths through the program are grouped into a finite set of classes. One path from each class is then tested.”‒Glossary of Computerized System and Software Development Terminology• Much more difficult than statement testing• For simple programs, can enumerate all paths through the code• Otherwise, sample paths through code with random input18Path Testing Example• Example pseudocode:• Requires four data sets:• condition1 is true and condition2 is true• condition1 is true and condition2 is false• condition1 is false and condition2 is true• condition1 is false and condition2 is false• Realistic program => combinatorial explosion!!!if
View Full Document
Unlocking...