Quality Management – Lessons of COQUALMO (COnstructive QUALity MOdel) A Software Defect Density Prediction ModelOutlineUSC Modeling MethodologyDelphi AssessmentAdding Project DataFig 11., Pg 170 SwCEwCII a posteiori Baysian UpdateSlide 7COQUALMO Model FrameworkSoftware Devel. Process(es) (cont.)Slide 10The Defect Introduction (DI) Sub-ModelA-Priori Expert-Judgment Based Code DI RangesDI Model EquationsInitial Data Analysis on the DI ModelSlide 15The Defect Removal (DR) Sub-ModelDefect Removal ProfilesSlide 18Static [Module-Level Code] AnalysisSlide 20Peer ReviewsSyntactic Versus Semantic CheckingExecution Testing and ToolsTechnique Selection GuidanceResidual Defects EquationDefect Densities from Expert- Judgment Calibrated COQUALMOValidation of Defect DensitiesSlide 28Slide 29Integrated COQUALMOCopyright 1999-2009 USC-CSSE1Quality Management – Lessons ofCOQUALMO (COnstructive QUALity MOdel)A Software Defect Density Prediction ModelAWBrown and Sunita Chulani, Ph.D.{AWBrown, sdevnani}@CSSE.usc.edu}USC-Center for Systems & Software Engineering (USC-CSSE)Copyright 1999-2009 USC-CSSE2OutlineBehavioral Underpinnings•Hidden Factory•Defect TypesCOQUALMO Framework•The Defect Introduction Sub-Model–Expert-Judgment Model + Some Initial Data Results•The Defect Removal Sub-Model–Expert-Judgment Model (Result of COQUALMO Workshop)•COQUALMO Integrated with COCOMO IICopyright 1999-2009 USC-CSSE3USC Modeling MethodologyCopyright 1999-2009 USC-CSSE4Delphi Assessment•Ask each expert the range for driver–Apply personal experience, –Look at completed projects, –Guess (WAG), •Collect and share in a meeting: discuss why/how different people made their estimate•Repeat until no changes•Final values (for each parameter)Max=(Hmax + 4*AVEmax + Lmax)/6Copyright 1999-2009 USC-CSSE5Adding Project Data •Effort Adjustment Multipliers (typical)–Linear RegressionCopyright 1999-2009 USC-CSSE6Fig 11., Pg 170 SwCEwCIIa posteiori Baysian Update•Used to combine expert judgement with sampled data: spread of datasets weights combinationCopyright 1999-2009 USC-CSSE7Outline•Model FrameworkThe Defect Introduction Sub-Model–Expert-Judgment Model + Some Initial Data Results•The Defect Removal Sub-Model–Expert-Judgment Model (Result of COQUALMO Workshop)•COQUALMO Integrated with COCOMO IICopyright 1999-2009 USC-CSSE8COQUALMO Model Framework• • •ResidualSoftwareDefectsCode DefectsRequirements DefectsDesign DefectsDefect Introduction pipesDefect Removal pipesCopyright 1999-2009 USC-CSSE9Software Devel. Process(es) (cont.): the hidden factoryCopyright 1999-2009 USC-CSSE10Outline•Model FrameworkThe Defect Introduction Sub-Model–Expert-Judgment Model + Some Initial Data Results•The Defect Removal Sub-Model–Expert-Judgment Model (Result of COQUALMO Workshop)•COQUALMO Integrated with COCOMO IICopyright 1999-2009 USC-CSSE11The Defect Introduction (DI) Sub-ModelSoftware Size estimateSoftware product, process, computer and personnel attributes(subset of COCOMO II factors)Defect IntroductionSub-ModelNumber of non-trivial requirements, design and code defects introducedCopyright 1999-2009 USC-CSSE12A-Priori Expert-Judgment Based Code DI Ranges1.00 1.50 2.00 2.50 3.00PMATRELYRESLCPLXPCAPPCONTOOLPRECPVOLLTEXSCEDTEAMSITEDOCUPEXPAEXPACAPDATATIMESTORRUSEFLEXCopyright 1999-2009 USC-CSSE13DI Model Equations•For each artifact j, Quality Adjustment Factor (QAF)QAF DIR - driveriji 122jj 13jBiji 121A (Size) (DI driver)j  •Estimated Number of Defects Introduced = j identifies the 3 artifact types (requirements, design and coding). A is the multiplicative calibration constant. B is initially set to 1 (DI-driver)ij is the Defect Introduction driver for the jth artifact and the ith factor.j 13jBjA (Size) QAFj •Estimated Number of Defects Introduced =Copyright 1999-2009 USC-CSSE14Initial Data Analysis on the DI ModelType ofArtifact1970’sBaselineDIRsQualityAdjustmentFactorPredictedDIRActual DIR CalibratedConstant(A)1990’sBaselineDIRsRequirements 5 0.5 2.5 4.5 1.8 9Design 25 0.44 11 8.4 0.77 19Code 15 0.5 7.5 16.6 2.21 33DIR = Defect Introduction RateCopyright 1999-2009 USC-CSSE15Outline•Model Framework•The Defect Introduction Sub-Model–Expert-Judgment Model + Some Initial Data ResultsThe Defect Removal Sub-Model–Expert-Judgment Model (Result of COQUALMO Workshop)•COQUALMO Integrated with COCOMO IICopyright 1999-2009 USC-CSSE16Defect Removal Sub-ModelDefect removal activity levelsNumber of non-trivial requirements, design and coding defects introducedNumber of residual defects/ unit of sizeSoftware Size EstimateThe Defect Removal (DR) Sub-ModelCopyright 1999-2009 USC-CSSE17Defect Removal Profiles•3 relatively orthogonal profiles–Automated Analysis–People Reviews–Execution Testing and Tools•Each profile has 6 levels–Very Low, Low, Nominal, High, Very High, Extra High•Very Low--removes the least number of defects •Extra High--removes the most defectsCopyright 1999-2009 USC-CSSE18Automated AnalysisRating Automated AnalysisVery Low Simple compiler syntax checking.LowBasic compiler or additional tools capabilities for static module-level codeanalysis, and syntax- and type-checking.Nominal All of the above, plusSome compiler extensions for static module and inter-module level codeanalysis, and syntax- and type-checking.Basic requirements and design consistency; and traceability checking.High All of the above, plusIntermediate-level module and inter-module code syntax and semantic analysis.Simple requirements/design consistency checking across views.Very High All of the above, plusMore elaborate requirements/design view consistency checking.Basic distributed-processing and temporal analysis, model checking, symbolicexecution.Extra High All of the above, plusFormalized* specification and verification.Advanced distributed-processing and temporal analysis, model checking,symbolic execution.*Consistency-checkable pre-conditions and post-conditions, but notmathematical theorems.Copyright 1999-2009 USC-CSSE19Static [Module-Level Code] Analysis"Static code analysis is the analysis of computer software that is performed without actually executing programs built from that software (analysis performed on executing programs is known as dynamic analysis). In most cases the analysis is performed on some version of the source code and in the other cases some form of the object code. "**