Slide 1OutlineMotivationMotivationMotivationTerms and DefinitionsThe ProblemProblems of InexperienceImprecise Project ScopingOverestimationUnderestimationProject Estimations Not RevisitedEstimations in ICSMManual Assessments are TediousSize ReportingLimitations in Software Cost EstimationOverstating Team’s CapabilitiesCompeting Project ProposalsUltimate ProblemThe GoalBenefitsOutlineIBM Self-Check [Kroll, 2008]Software Sizing and EstimationProject Tracking and AssessmentOutlineProject Tracking FrameworkSize CountingProject Tracking ResultsTeam Assessment FrameworkQuestion DevelopmentAdjusting the COCOMOII EstimatesExample ScenarioOutlineCOCOMO II Estimation RangeCSCI577 Estimation ErrorsConclusionOutlineTool Support for FrameworkTool SupportDifferent Project TypesToolToolToolToolPublicationsReferencesReferencesReferencesUniversity of Southern CaliforniaCenter for Systems and Software EngineeringShrinking the Cone of Uncertainty with Continuous AssessmentPongtip AroonvatanapornCSCI 510Fall 2011October 5, 201110/5/11 (C) USC-CSSE 1University of Southern CaliforniaCenter for Systems and Software EngineeringOutline•Introduction–Motivation–Problems•Related Works•Proposed Methodologies•Conclusion•Tool Demo10/5/11 (C) USC-CSSE 2University of Southern CaliforniaCenter for Systems and Software EngineeringMotivation•The Cone of Uncertainty•Exists until the product is delivered, or even after•The wider, the more difficult to ensure accuracies and timely deliveries•Focus on uncertainties of team aspects from product design onwards–COCOMO II space–Many factors before that (requirements volatility, technology, etc.)10/5/11 3(C) USC-CSSEUniversity of Southern CaliforniaCenter for Systems and Software EngineeringMotivation•Key principles of ICSM–Stakeholder satisficing–Incremental and evolutionary growth of system definition and stakeholder commitment–Iterative system development and definition–Concurrent system definition and development–Risk management•COCOMO II–COCOMO II space is in the development cycle–Influences on estimations and schedules [Construx, 2006]•Human factors: 14x•Capability factors: 3.5x•Experience factors: 3.0x10/5/11 4(C) USC-CSSEUniversity of Southern CaliforniaCenter for Systems and Software EngineeringMotivation•Standish CHAOS Summary 2009•Surveyed 9000 projects10/5/11 5Delivered with full capability within budget and schedule 32%Cancelled 24%Over budget, over schedule, or undelivered 44%68% project failure rate(C) USC-CSSEUniversity of Southern CaliforniaCenter for Systems and Software EngineeringTerms and Definitions•Inexperienced–Inexperienced in general–Experienced, but in new domain–Experienced, but using new technology•Continuous Assessment–Assessments take place over periods of time–Done in parallel with process, instead of only at the end–Widely used in education–Used in software process measurement [Jarvinen, 2000]10/5/11 6(C) USC-CSSEUniversity of Southern CaliforniaCenter for Systems and Software EngineeringThe Problem•Experienced teams can produce better estimates–Use “yesterday’s weather”–Past projects of comparable sizes–Past data of team’s productivity–Knowledge of accumulated problems and solutions•Inexperienced teams do not have this luxuryNo tools or data that monitors project’s progression within the cone of uncertaintyNo tools or data that monitors project’s progression within the cone of uncertainty10/5/11 7(C) USC-CSSEUniversity of Southern CaliforniaCenter for Systems and Software EngineeringProblems of Inexperience•Imprecise project scoping–Overestimation vs. underestimation•Project estimations often not revisited–Insufficient data to perform predictions–Project’s uncertainties not adjusted•Manual assessments are tedious–Complex and discouraging•Limitations in software cost estimation–Models cannot fully compensate for lack of knowledge and understanding of project•Overstating team’s capabilities–Unrealistic values that do not reflect project situation–Teams and projects misrepresented (business vs. technical)10/5/11 8(C) USC-CSSEUniversity of Southern CaliforniaCenter for Systems and Software EngineeringImprecise Project Scoping•Based on CSCI577 data, projects either significantly overestimate or underestimate effort–Possibly due to: •Unfamiliarity with COCOMO•Inexperienced•Teams end up with inaccurate project scoping–Promise too much10/5/11 (C) USC-CSSE 9University of Southern CaliforniaCenter for Systems and Software EngineeringOverestimation•Estimate is too high to achieve within available resources•Need to reduce the scope of the project–Re-negotiate requirements with client–Throw away some critical core capabilities•Lose the expected benefits•Often do not meet client satisfactions/needs10/5/11 (C) USC-CSSE 10University of Southern CaliforniaCenter for Systems and Software EngineeringUnderestimation•Estimate is lower than actual•Project appears that it can be done in with less resources–Clients may ask for more capabilities–Teams may end up promising more•As project progresses, team may realize that project is not achievable–If try to deliver what was promised, quality suffers–If deliver less that what was promised, clients suffer10/5/11 (C) USC-CSSE 11University of Southern CaliforniaCenter for Systems and Software EngineeringProject Estimations Not Revisited•At the beginning, teams do not always have the necessary data–No “yesterday’s weather”–High number of uncertainties•Initial estimates computed are not accurate•If estimates are readjusted, no problem•Reality is, estimates are left untouched10/5/11 (C) USC-CSSE 12University of Southern CaliforniaCenter for Systems and Software EngineeringEstimations in ICSM•Estimates are “supposedly” adjusted during each milestone reviews–Reviewed by team–Reviewed by stakeholders•Adjustments require necessary assessments to become more accurate•Without assessments, adjustments are made with no directions10/5/11 (C) USC-CSSE 13University of Southern CaliforniaCenter for Systems and Software EngineeringManual Assessments are Tedious•Complex process•Time consuming•Require experienced facilitator/assessor to perform effectively•Often done by conducting various surveys, analyze the data, and determine weak/strong points–Repeated as necessary•Discouraging to the teams10/5/11 (C) USC-CSSE 14University of