Today’s AgendaOutlineSoftware EngineeringSoftware TestingSlide 5Fault, Error & Failure (1)Fault, Error, and Failure (2)Fault, Error, and Failure (3)Fault & Failure ModelStatic & Dynamic TestingTest CaseTesting & DebuggingVerification & ValidationQuality AttributesTestabilitySlide 16The ProcessTest & Debug CycleAn ExampleTest planTest DataTest HarnessTest OracleSlide 24Classifier C1: Source of Test GenerationClassifier C2: Life Cycle PhasesClassifier C3: Goal Directed TestingClassifier C4: Artifact Under TestSlide 29PhilosophySlide 31SummaryToday’s AgendaHW #1Finish IntroductionInput Space PartitioningSoftware Testing and Maintenance 1Software Testing and Maintenance 2Outline Introduction Basic Concepts  The Testing Process Types of Testing Testing Philosophy SummarySoftware Testing and Maintenance 3Software Engineering Software has become pervasive in modern societyDirectly contributes to quality of lifeMalfunctions cost billions of dollars every year, and have severe consequences in a safe-critical environment How to build quality software, especially for large-scale development?Requirements, design, coding, testing, maintenance, configuration, documentation, deployment, and etc.Software Testing and Maintenance 4Software TestingSteve Ballmer, 2006: “Let's acknowledge a sad truth about software: any code of significant scope and power will have bugs in it.”A dynamic approach to detecting software faultsAlternatively, static analysis can be performed, which is however often intractableInvolves sampling the input space, running the test object, and observing the runtime behaviorThe single most widely used approach in practiceLabor intensive, and often consumes more than 50% of development costSoftware Testing and Maintenance 5Outline Introduction Basic Concepts  The Testing Process Types of Testing Testing Philosophy SummarySoftware Testing and Maintenance 6Fault, Error & Failure (1) Fault : A static defect in the softwareIncorrect instructions, missing instructions, extra instructions Error : An incorrect internal state that is the manifestation of some fault Failure : External, incorrect behavior with respect to the requirements or other description of the expected behaviorSoftware Testing and Maintenance 7Fault, Error, and Failure (2)public static int numZero (int[] x) { // effects: if x == null throw NullPointerException // else return the number of occurrences of 0 in x int count = 0; for (int i = 1; i < x.length; i ++) { if (x[i] == 0) { count ++; } } return count;}Software Testing and Maintenance 8Fault, Error, and Failure (3) The state of numZero consists of the values of the variables x, count, i, and the program counter. Consider what happens with numZero ([2, 7, 0]) and numZero ([0, 7, 2])?Software Testing and Maintenance 9Fault & Failure Model Three conditions must be satisfied for a failure to be observedReachability : The location or locations in the program that contain the fault must be reachedInfection : The state of the program must be incorrectPropagation : The infected state must propagate to cause some output of the program to be incorrectSoftware Testing and Maintenance 10Static & Dynamic Testing Static Testing: Testing without executing the program.Code walkthrough & inspection, and various static analysis techniques. Dynamic Testing: Testing by executing the program with real inputsStatic information can often be used to make dynamic testing more efficient.Software Testing and Maintenance 11Test Case Test data: data values to be input to the program under test Expected result: the outcome expected to be produced by the program under testSoftware Testing and Maintenance 12Testing & Debugging Testing: Finding inputs that cause the software to fail Debugging: The process of finding a fault given a failure In practice, testing & debugging are often performed in a cyclic fashionSoftware Testing and Maintenance 13Verification & Validation Verification: Ensure compliance of a software product with its design  Validation: Ensure compliance of a software product with intended usage Question: Which task, validation or verification, is more difficult to perform?Software Testing and Maintenance 14Quality Attributes Static attributes refer to the actual code and related documentationWell-structured, maintainable, and testable codeCorrect and complete documentation Dynamic attributes refer to the behavior of the application while in useReliability, correctness, completeness, consistency, usability, and performanceSoftware Testing and Maintenance 15Testability The degree to which a system or component facilitates the establishment of test criteria and the performance of tests to determine whether those criteria have been met The more complex an application, the lower the testability, i.e., the more effort required to test it Design for testability: Software should be designed in a way such that it can be easily testedSoftware Testing and Maintenance 16Outline Introduction Basic Concepts  The Testing Process Types of Testing Testing Philosophy SummarySoftware Testing and Maintenance 17The Process Preparing a test plan Constructing test data Executing the program Specifying program behavior Evaluating program behavior Construction of automated oraclesSoftware Testing and Maintenance 18Test & Debug CycleConstruct test inputExecute programIs behavior as expected?Testing to be terminated?StopOperational profileTest planSpecificationCause of error to be determined now?Debug the programError to be fixed now?File pending error reportFix ErrorInput domaininput dataTest casebehavioryesnoyesuseuseusenoSoftware Testing and Maintenance 19An ExampleProgram sort:  Given a sequence of integers, this program sorts the integers in either ascending or descending order. The order is determined by an input request character “A” for ascending or “D” for descending.Software Testing and Maintenance 20Test plan1. Execute the program on at least two input sequences, one with “A” and the other with “D” as request characters2. Execute the program on an empty input sequence3. Test the program for robustness against invalid inputs such as “R” typed in as the request character4. All failures of the test program