Slide 1TestingDefinition - TestingGoals for TestingTasks associated with TestingTerms used in TestingSoftware ReliabilityTypes of Test TechniquesStatic TechniquesDynamic TechniquesTest LevelsTest Levels Continued…Regression TestObjectives of TestingStrategies for Test Case SelectionOther Test CriteriaTest-First ProgrammingTest ExecutionTest DocumentationTest ManagementTest MeasuresConclusionsSlide 23A Brief Essay on Software Testing-Antonia Bertolino, Eda MarchettiPresented by Gargi Chipalkatti(Software Engineering II - EEL 6883)Testing•Testing refers to many different activities used to check a piece of software•Even after successful completion of an extensive testing, the software can still contain faults•Testing can never prove the absence of defects, it can only possibly reveal the presence of faults by provoking malfunctionsDefinition - Testing•Definition– Software Testing consists of the dynamic verification of the behavior of a program on a finite set of test cases, suitably selected from the usually infinite executions domain, against the specified expected behavior.•Explanation– Dynamic – means that testing implies executing the program on (valued) inputs– Finite – means that only a limited number of test cases can be executed during the testing phase, chosen from the whole test set, that can generally be considered infinite– Selected – refers to the test techniques adopted for selecting the test cases (and testers must be aware that different selection criteria may yield vastly different effectiveness)– Expected – points out to the decision process adopted for establishing whether the observed outcomes of program execution are acceptable or notGoals for Testing•Detection of “bugs” in the software•Increase confidence in the proper functioning of the software•Assist with the evaluation of functional and nonfunctional properties (as well as performance characteristics)•Exposing potential design flaws•Exposing deviations from user’s requirements•Measuring the operational reliability•Producing estimates of software reliabilityTasks associated with Testing•Deriving an adequate suite of test cases, according to a feasible and cost-effective test selection technique•The ability to launch the selected tests (in a controlled host environment, or worse in the tight target environment of an embedded system)•Deciding whether the test outcome is acceptable or not (which is referred to as the test oracle problem)•Evaluating the impact of the failure and finding its direct cause (the Fault), and the indirect one (via Root Cause Analysis)•Judging whether testing is sufficient and can be stopped•Measuring the effectiveness of the testsTerms used in Testing•Definitions–Fault – The cause of a failure, e.g., a missing or incorrect piece of code–Error – An intermediate unstable state–Failure – The manifested inability of the program to perform the function required, i.e., a system malfunction evidenced by incorrect output, abnormal termination or unmet time and space constraints.•Relationship–A fault may remain undetected long time, until some event activates it. –If an when the error propagates to the output, it eventually causes the failure.Fault  Error  Failure –This chain can recursively iterate: a fault in turn can be caused by the failure of some other interacting system.Software Reliability•Software reliability is a probabilistic estimate, and measures the probability that the software will execute without failure in a given environment for a given period of time.•Thus, the value of software reliability depends on how frequently those inputs that cause a failure will be exercised by the final users.•The notion of reliability is specific to “a given environment”, the tests must be drawn from an input distribution that approximates as closely as possible the future usage in operation, which is called the operational distribution.Types of Test TechniquesStatic Analysis Techniques ––Based solely on the (manual or automated) examination of project documentation of software models and cod, and of other related information about requirements and design–Generally yields valid results, but may be weak in precisionDynamic Test Techniques ––Exercise the software in order to expose possible failures–Behavioral and performance properties of the program are also observed–Yields more precise results, but only holding for the examined executionsStatic Techniques• Can be applied at the requirements phase; at the design phase; and during the implementation phase•Traditional Static Techniques – heavily manual, error-prone, time consuming–Software inspection – The step-by-step analysis of the documents (deliverables) produced, against a compiled checklist of common and historical defects–Software reviews – The process by which different aspects of the work product are presented to project personnel (managers, users, customer etc) and other interested stakeholders for comment or approval–Code reading – The desktop analysis of the produced code for discovering typing errors that do not violate style or syntax–Algorithm analysis and tracing – The process in which the complexity of algorithms employed and the worst case, average-case and probabilistic analysis evaluations can be derived•Static Analysis Techniques relying on use of Formal MethodsDynamic Techniques•Dynamic Techniques ––Testing – Based on the execution of the code on valued inputs (definition of the parameters and environmental conditions that characterize a system state must be included when necessary )–Profiling – A program profile records the number of times some entities of interest occur during a set of controlled executionsTest Levels•Unit Test ––A unit is the smallest testable piece of software, which may consist of hundreds or even just a few lines of source code, and generally represents the result of the work of one programmer–The purpose is to ensure that the unit satisfies its functional specification and/or that its implemented structure matches the intended design structure •Integration Test ––Integration is the process by which software pieces or components are aggregated to create a larger component.–Specifically aimed at exposing the problems that can arise at this stage.–Strategies – Big-Bang, Top-Down, Bottom-UpTest Levels Continued…•System Test