Introduction to Software Testing Chapter 3 1 3 2 Logic Coverage Paul Ammann Jeff Offutt http www cs gmu edu offutt softwaretest Ch 3 Logic Coverage Four Structures for Modeling Software Graphs Logic Input Space Syntax Applied to Applied to Source Specs Source Specs Design Introduction to Software Testing Ch 3 Applied to FSMs DNF Source Models Integ Use cases Ammann Offutt Input 2 1 Covering Logic Expressions 3 1 Logic expressions show up in many situations Covering logic expressions is required by the US Federal Aviation Administration for safety critical software Logical expressions can come from many sources Decisions in programs FSMs and statecharts Requirements q Tests are intended to choose some subset of the total number of truth assignments to the expressions Introduction to Software Testing Ch 3 Ammann Offutt 3 Logic Predicates and Clauses A predicate is an expression that evaluates to a boolean value Predicates can contain boolean variables non boolean variables that contain boolean function calls Internal structure is created by logical operators the negation operator the and operator the or operator the implication operator the exclusive or operator the equivalence operator A clause is a predicate with no logical operators Introduction to Software Testing Ch 3 Ammann Offutt 4 2 Examples a b f z D m n o Four clauses a b relational expression f z boolean valued b l l d function f ti D boolean variable m n o relational expression Most predicates have few clauses It would be nice to quantify that claim Sources of predicates Decisions in p programs g Guards in finite state machines Decisions in UML activity graphs Requirements both formal and informal SQL queries Introduction to Software Testing Ch 3 5 Ammann Offutt Translating from English I am interested in SWE 637 and CS 652 course swe637 OR course cs652 Humans have trouble translating from English to Logic If you leave before 6 30 AM take Braddock to 495 if you leave after 7 00 AM take Prosperity to 50 then 50 to 495 time 6 30 path Braddock time 7 00 path Prosperity Hmm this is incomplete time 6 30 path Braddock time 6 30 path Prosperity Introduction to Software Testing Ch 3 Ammann Offutt 6 3 Testing and Covering Predicates 3 2 We use predicates in testing as follows Developing p g a model of the software as one or more p predicates Requiring tests to satisfy some combination of clauses Abbreviations P is the set of predicates p is a single predicate in P C is the set of clauses in P Cp is the set of clauses in predicate p c is a single clause in C Introduction to Software Testing Ch 3 Ammann Offutt 7 Predicate and Clause Coverage The first and simplest two criteria require that each predicate and each clause be evaluated to both true and false Predicate Coverage PC For each p in P TR contains two requirements p evaluates to true and p evaluates to false When predicates come from conditions on edges this is equivalent to edge coverage PC does not evaluate all the clauses so Clause Coverage CC For each c in C TR contains two requirements c evaluates to true and c evaluates to false Introduction to Software Testing Ch 3 Ammann Offutt 8 4 Predicate Coverage Example a b D m n o predicate coverage Predicate true a 5 b 10 D true m 1 n 1 o 1 5 10 true 1 1 1 true true TRUE true P di t false Predicate f l a 10 b 5 D false m 1 n 1 o 1 10 5 false 1 1 1 false false TRUE false 9 Ammann Offutt Introduction to Software Testing Ch 3 Clause Coverage Example a b D m n o Clause coverage a b true a b false D true D false a 5 b 10 a 10 b 5 D true m n o true D false m n o false m 1 n 1 o 1 m 1 n 2 o 2 false cases Two tests true cases 1 a 5 b 10 D true m 1 n 1 o 1 2 a 10 b 5 D false m 1 n 2 o 2 Introduction to Software Testing Ch 3 Ammann Offutt 10 5 Problems with PC and CC PC does not fully exercise all the clauses especially in the presence of short circuit evaluation p CC does not always ensure PC That is we can satisfy CC without causing the predicate to be both true and false This is definitely not what we want The simplest solution is to test all combinations Ammann Offutt Introduction to Software Testing Ch 3 11 Combinatorial Coverage CoC requires every possible combination Sometimes called Multiple Condition Coverage Combinatorial Coverage g CoC For each p in P TR has test requirements for the clauses in Cp to evaluate to each possible combination of truth values a b D 1 T 2 T 3 T 4 T 5 F 6 F 7 F 8 F Introduction to Software Testing Ch 3 T T F F T T F F m n o T F T F T F T F a b D m n o T F T F T F F F Ammann Offutt 12 6 Combinatorial Coverage This is simple neat clean and comprehensive But quite expensive 2N tests where N is the number of clauses Impractical for predicates with more than 3 or 4 clauses The literature has lots of suggestions some confusing The general idea is simple Test each clause independently from the other clauses Getting the details right is hard What exactly does independently mean The book presents this idea as making clauses active Introduction to Software Testing Ch 3 Ammann Offutt 13 Active Clauses Clause coverage has a weakness The values do not always make a difference Consider the first test for clause coverage which caused each clause to be true 5 10 true 1 1 1 Only the first clause counts To really test the results of a clause the clause should be the determining factor in the value of the predicate Determination A clause c in predicate p called the major i clause determines p if and only if the values of the remaining minor clauses cj are such that changing ci changes the value of p This is considered to make the clause active Introduction to Software Testing Ch 3 Ammann Offutt 14 7 Determining Predicates P A B P A B if B true p is always true if B false p is always false so if B false false A determines p p so if B true true A determines p p if A false B determines p if A true B determines p Goal Find tests for each clause when the clause determines the value of the predicate This Thi is i formalized f li d in i severall criteria it i that th t h have subtle btl but b t very important differences Ammann Offutt Introduction to Software Testing Ch 3 15 Active Clause Coverage Active Clause Coverage ACC For each p in P and …