Value-Based Software Engineering: Motivation and Key PracticesOutlineSoftware Testing Business CaseSlide 4Slide 520% of Features Provide 80% of Value: Focus Testing on These (Bullock, 2000)Value-Based Testing Provides More Net ValueSlide 8Motivation for Value-Based SEWhy Software Projects FailSlide 1220% of Fires Cause 80% of Property Loss: Focus Fire Dispatching on These?Missing Stakeholder Concerns: Fire Dispatching SystemKey DefinitionsConclusions So FarSlide 17Slide 18Slide 19Slide 20Projecting Yourself Into Others’ Win SituationsSlide 22EasyWinWin OnLine Negotiation StepsSlide 24Slide 25Slide 26Examples of Utility Functions: Response TimeHow Much Testing is Enough? - Early Startup: Risk due to low dependability - Commercial: Risk due to low dependability - High Finance: Risk due to low dependability - Risk due to market share erosionValue-Based Defect Reduction Example: Goal-Question-Metric (GQM) ApproachValue-Based GQM Approach – IValue-Based GQM Approach – IIValue-Based GQM ResultsSlide 33Is This A Risk?Slide 35Earlier, This Problem Was A RiskHow Can Risk Management Help You Deal With Risks?Risk Management Strategies: - Buying InformationOther Risk Management StrategiesIs Risk Management Fundamentally Negative?What Else Can Risk Management Help You Do?Example Large-System Risk Analysis: How Much Architecting is Enough?How Soon to Define Subcontractor Interfaces? Risk exposure RE = Prob(Loss) * Size(Loss) -Loss due to rework delaysSlide 44Slide 45Slide 46Slide 47Slide 48Slide 49Change As Opportunity: Agile MethodsFive Critical Decision FactorsSlide 52Slide 53Slide 54University of Southern CaliforniaCenter for Software Engineering C S E USCBarry Boehm, USCCS 510 Lecture Fall 2009Value-Based Software Engineering:Motivation and Key PracticesUniversity of Southern CaliforniaCenter for Software Engineering C S E USC07/08/09 2Outline•Value-based software engineering (VBSE) motivation, examples, and definitions•VBSE key practices–Benefits realization analysis–Stakeholder Win-Win negotiation–Business case analysis–Continuous risk and opportunity management–Concurrent system and software engineering–Value-based monitoring and control–Change as opportunity•Conclusions and references©USC-CSSEUniversity of Southern CaliforniaCenter for Software Engineering C S E USC07/08/093Software Testing Business Case•Vendor proposition–Our test data generator will cut your test costs in half–We’ll provide it to you for 30% of your test costs–After you run all your tests for 50% of your original cost, you are 20% ahead•Any concerns with vendor proposition?©USC-CSSEUniversity of Southern CaliforniaCenter for Software Engineering C S E USC07/08/094Software Testing Business Case•Vendor proposition–Our test data generator will cut your test costs in half–We’ll provide it to you for 30% of your test costs–After you run all your tests for 50% of your original cost, you are 20% ahead•Any concerns with vendor proposition?–34 reasons in 2004 ABB experience paper•Unrepresentative test coverage, too much output data, lack of test validity criteria, poor test design, instability due to rapid feature changes, lack of preparation and experience (automated chaos yields faster chaos), …•C. Persson and N. Yilmazturk, Proceedings, ASE 2004–But one more significant reason©USC-CSSEUniversity of Southern CaliforniaCenter for Software Engineering C S E USC07/08/09 5Software Testing Business Case•Vendor proposition–Our test data generator will cut your test costs in half–We’ll provide it to you for 30% of your test costs–After you run all your tests for 50% of your original cost, you are 20% ahead•Any concerns with vendor proposition?–Test data generator is value-neutral*–Assumes every test case, defect is equally important–Usually, 20% of test cases cover 80% of business case* As are most current software engineering techniques©USC-CSSEUniversity of Southern CaliforniaCenter for Software Engineering C S E USC07/08/09 620% of Features Provide 80% of Value: Focus Testing on These (Bullock, 2000)% of Valuefor CorrectCustomerBillingCustomer Type100806040205 10 15Automated test generation tool - all tests have equal value©USC-CSSEUniversity of Southern CaliforniaCenter for Software Engineering C S E USC07/08/09 7Value-Based Testing Provides More Net ValueNet Value NV6040200-2020 40 10060 80-40(100, 20)Percent of tests runTest Data GeneratorValue-Based Testing(30, 58)% Tests Test Data Generator Value-Based TestingCost Value NV Cost Value NV0 30 0 -30 0 0 010 35 10 -25 10 50 4020 40 20 -20 20 75 5530 45 30 -15 30 88 5840 50 40 -10 40 94 54…. …. …. …. …. …. ….100 80 100 +20 100 100 0©USC-CSSEUniversity of Southern CaliforniaCenter for Software Engineering C S E USC07/08/09 8By Number P-value % Gr A higher By Impact P-value % Gr A higherAverage of Concerns0.202 34Average Impact of Concerns0.049 65Average of Problems0.056 51Average Impact of Problems0.012 89Average of Concerns per hour0.026 55Average Cost Effectiveness of Concerns0.004 105Average of Problems per hour0.023 61Average Cost Effectiveness of Problems0.007 108• Group A: 15 IV&V personnel using VBR procedures and checklists• Group B 13 IV&V personnel using previous value-neutral checklists– Significantly higher numbers of trivial typo and grammar faultsValue-Based Reading (VBR) Experiment— Keun Lee, ISESE 2005©USC-CSSEUniversity of Southern CaliforniaCenter for Software Engineering C S E USC07/08/09 9Motivation for Value-Based SE•Current SE methods are basically value-neutral–Every requirement, use case, object, test case, and defect is equally important–Object oriented development is a logic exercise–“Earned Value” Systems don’t track business value–Separation of concerns: SE’s job is to turn requirements into verified code–Ethical concerns separated from daily practices•Value – neutral SE methods are increasingly risky–Software decisions increasingly drive system value–Corporate adaptability to change achieved via software decisions–System value-domain problems are the chief sources of software project failures©USC-CSSEUniversity of Southern CaliforniaCenter for Software Engineering C S E USC07/08/09 10Why Software Projects Fail©USC-CSSEUniversity of Southern CaliforniaCenter for Software Engineering C S E USC07/08/09 11 The “Separation of Concerns” Legacy•“The notion of