The Organizational Impacts on Software Quality and Defect EstimationDiscussion TopicsIntroductionSlide 4Defect Concepts: What is a Defect?Slide 6Slide 7Defect Concepts: Static Vs. Dynamic DiscoveryDefect Estimation MethodsDefect Estimation Methods Linear RegressionDefect Estimation Methods Capture/RecaptureDefect Estimation Methods AI: Bayesian Belief NetworksDefect Estimation Methods AI: Neural NetworksDefect Estimation VariablesFurther ResearchConclusionQuestionsThe Organizational Impacts on Software Quality and Defect EstimationStephen Lopez-CoutoDiscussion TopicsIntroductionDefect ConceptsDefinition of a defectStatic vs. Dynamic DiscoveryDefect Estimation MethodsLinear RegressionCapture/RecaptureAIDefect Estimation VariablesFurther ResearchConclusionIntroductionDefect EstimationThe act of “guessing” the number of defects that exist in a current software baselinePurpose of this paper is to determine the usefulness of metrics that are not explicitly tied to the code to make the estimatesIntroductionOrganizational Elements:Development Team SizeDeveloper ExperienceInstitutional Processes CMMI level for exampleDevelopment ToolsDevelopment ScheduleProgramming LanguageSoftware ArchitectureDefect Concepts: What is a Defect?There is no common definition of a defectConsider the following:Program is supposed to add the variables ‘y’ and ‘i’ together and put the value into ‘x’int i = 5;int y = 6;int x = y+1Print(x);Defect Concepts: What is a Defect?There is no common definition of a defectConsider the following:Program is supposed to add the variables ‘y’ and ‘i’ together and put the value into ‘x’int i = 5;int y = 6;int x = y+1Print(x);The programmer introduced a defectCorrect code: int x = y + iDefect Concepts: What is a Defect?Two things occur before a defect can be detected1. Fault – Incorrect value in the internal state of the program2. Failure – When a fault is realized as outputint i = 5;int y = 6;int x = y+1 // Causes a FaultPrint(x); // Causes a FailureA Failure may not be detected until well after the fault has occurred!Defect Concepts: Static Vs. Dynamic DiscoveryStatic – Does not utilize executing codeSoftware InspectionsWalkthroughsComplexity MappingDynamic – Utilizes executing, compiled codeRuntime testsAutomated defect discovery toolsDefect Estimation MethodsEstimating the number of defects is by no means an exact scienceThere are lots of different ways to determine an estimateMost of the methods are highly tied to a specific organization or software baselineNot good enough for general useThree general methods will be discussedLinear RegressionCapture/RecaptureArtificial IntelligenceBayesian Belief NetworksNeural NetworksDefect Estimation MethodsLinear RegressionMethod:Determines a mathematical expression that relates some number of independent variables (the input metrics) and the dependent variable (defect estimation)Variable Type: Numeric OnlyDefect Estimation MethodsCapture/RecaptureMethod:Some number of defects are seeded into the software at development time. When testing occurs ratio of seeded to unseeded defects found is used to estimate the total number. Variable Type: N/A. Only track defect counts.Defect Estimation MethodsAI: Bayesian Belief NetworksMethod:A network that weighs the relationship among different variables is created (using expert knowledge) and processed using Bayesian probabilities to determine the quality of the software.Variable Type: anyDefect Estimation MethodsAI: Neural NetworksMethod:A multilayer perceptron using the back propagation algorithm is trained on legacy defect data and then provides estimates based on input data. Variable Type: Numeric OnlyDefect Estimation VariablesFurther ResearchThree main areas1. Transformation of non numerical data into a numerical form2. Additional estimation methods that do not utilize numeric only data3. Determination of common relationships among the input metrics, independent of which method is usedConclusionOrganizational Element Data is useful when paired with the proper estimation methodLimitations of the most common method (linear regression) has led to a general shunning of these metricsDefect Estimation approaches are not ready for real world useThey are too tied to specific