The Software Engineering Laboratory (SEL)Observation, Feedback, Learning, PackagingSEL Experience Factory StructureTechnology Transition ProcessImproving via the Experience FactoryUsing Baselines to Show Improvement1987 vs. 1991The Software Engineering Laboratory (SEL)Consortium ofNASA/GSFCComputer Sciences CorporationUniversity of MarylandEstablished in 1976Goals- better understand software development- improve the process and product qualityat Goddard, formerly in the Flight Dynamics Division, now at theInformation Systems Centerusing observation, experimentation, learning, and model buildingObservation, Feedback, Learning, PackagingUsed the SEL as a laboratory to build models, test hypotheses, Observation played a key roleFeedback loops provided an environment for learningGenerated lessons learned that were packaged into process, product and organizational structureUsed the University to test high risk ideasDeveloped technologies, methods and theories when necessaryLearned what worked and didn’t work, applied ideas when applicableKept the business going with an aim at improvement, learningSEL Experience Factory StructureDEVELOPERS(SOURCE OF EXPERIENCE)(PACKAGE EXPERIENCE FOR REUSE)DATA BASE SUPPORT(MAINTAIN/QA EXPERIENCE INFORMATION)Development measures for each projectRefinements to development processSTAFF 275-300 developersTYPICAL PROJECT SIZE 100-300 KSLOCACTIVE PROJECTS 6-10 (at any given time)PROJECT STAFF SIZE 5-25 peopleTOTAL PROJECTS(1976-1994) 120STAFF 10-15 AnalystsFUNCTION • Set goals/questions/metrics- Design studies/experiments• Analysis/Research• Refine software process- Produce reports/findingsPRODUCTS(1976-1994) 300 reports/documentsSEL DATA BASEFORMS LIBRARYREPORTS LIBRARY160 MB220,000• SEL reports• Project documents• Reference papersSTAFF 3-6 support staffFUNCTION • Process forms/data•QA all data• Record/archive data• Maintain SEL data base• Operate SEL libraryNASA + CSC + U of MDNASA + CSC NASA + CSC POPROCESS ANALYSTSEFTechnology Transition ProcessImproving via the Experience FactoryExperience FactoryProject OrganizationResearcher AnalystExperimenter TeamModel PackagerProjects 2, 3, . . .Project 1public domain processproblemscurrent local processtailored processlessons learned & recommended changesSEL tailored processgoalsmeasurement & feedbackUsing Baselines to Show Improvement1987 vs. 1991Error Rates (development)0246810Early Baseline 8 similar systemsCurrent 7 similar systemsErrors/KLOC (developed)Average ~4.5Average ~1Low 1.7Low 0.2High 2.4High 8.90200400600800Cost (staff months)Early Baseline 8 similar systems supporting 4 projectsCurrent 7 similar systems supporting 4 projectsStaff monthsAverage ~490Average ~210Low 357High 755Low 98High 277ReuseEarly Baseline 8 similar systemsCurrent 8 similar systems% ReuseFORTRAN (3 systems)Ada (5 systems)020406080100Average ~79%6190IEEE39Average ~20%Early Baseline = 1985-1989 Current = 1990-1993Decreased 75%Reduced 55%Increased 300%Using Baselines to Show ImprovementContinuous Improvement in the SELDecreased Development Defect rates by 75% (87 - 91) 37%(91 - 95)Reduced Cost by 55% (87 - 91) 42% (91 - 95)Improved Reuse by 300% (87 - 91) 8% (91 - 95)Increased Functionality five-fold (76 - 92)CSC officially assessed as CMM level 5 and ISO certified (1998), starting with SEL organizational elements and