8.1The Experience Factory:Packaging Software ExperienceSince software deals with man-made artifacts, we need to view software development as anexperimental science and build models of the artifacts and the processes by which they aremanufactured. To do this we need to isolate and categorize the components of the discipline, definenotations for representing these components, and specify the interrelationships among these componentsas they are manipulated. The components of the discipline consist of various processes (e.g., life cyclemodels, methods, techniques, tools), products (e.g., code components, requirements, designs,specifications, test plans), and other forms of experience (e.g., resource models, defect models, qualitymodels, economic models).We need to build descriptive models of the discipline components to better understand (1) the nature ofthe processes and products and their various characteristics, (2) the variations among them, (3) theweaknesses and strengths of both, and (4) mechanisms to predict and control them. Models exist forsome components, for example there are several mathematical models of programs and modules, thereare parametrized cost models that attempt to predict the cost of a project based upon past experience,there are informal descriptions of the life cycle process. However many more models are needed andthose models that do exist need to be more formally defined and further analyzed and integrated toprovide a deeper understanding of the components and their interactions.Based upon analysis of these descriptive models, we need to build prescriptive models that improveboth products and and the processes for creating them relative to a variety of qualities, providefeedback for project control, and allow the packaging of successful experience. Because the overallsolutions are both technical and managerial, model building requires the support of a variety ofdisciplines both within and outside the discipline.The overall approach requires an approach, similar to the scientific method, that allows us toexperiment, measure, learn, build better models, and reuse past experiences. For the past 15 years, wehave been applying such an approach, the Quality Improvement Paradigm,in the Software EngineeringLaboratory (SEL) at NASA/GSFC. It adapts the scientific method to software development. As statedin the first chapter, the basic steps involve:Planning: an iterative process involving characterizing the current project and its environment, setting thequantifiable goals for successful project performance and improvement, and choosing the appropriateprocess model and supporting methods and tools for this project.Execution: a closed-loop project cycle which involves executing the processes, constructing theproducts, collecting and validating the prescribed data ,and analyzing it in real-time to pride feedbackfor corrective action on the current project.8.2Analysis and Packaging: a post mortem analysis of the data and information gathered to evaluate thecurrent practices, determine problems, record findings, and make recommendations for future projectimprovements, and a packaging of the experience gained in the form of updated and refined models andother forms of structured knowledge gained from this and prior projects and the storing of thepackages in an experience base so it is available for future projects.The Software Engineering Laboratory ExperienceIn the evolution of the SEL, we can identify three three major phases, understanding, assessing andimproving and packaging. During the first phase, we worked on understanding the environment andhow to measure it. To achieve this, we measured what we could, e.g., resources and defects, appliedwhatever models existed, e.g., cost models and reliability models, built baselines for such things asdefect classes and resource allocation, and developed the Goal/Question/Metric paradigm as anorganized mechanism for setting goals and measuring the software project.During this phase we learned that although there are similarities among project environments, thedifferences require that each project be treated differently, i..e., different processes, methods andtechniques need to be tailored to the specific needs of the project. Thus there is not one standard lifecycle process for all software projects. We learned that there is a direct relationship between theprocesses performed and the various product qualities. Thus, one can effect specific product qualitiesby making specific changes in the process. We learned that measurement needs to be based upon goalsand models. Performing measurement bottom-up neither provides the right metrics nor the means forinterpreting them. We learned that evaluation and feedback are necessary for project control, i.e.,providing managers with the right data helps them manage the project in a more effective way.In phase two, we worked to improve the process and product quantitatively based upon theevolutionary development of various models. To this end, we experimented with various reading andtesting technologies (reading by stepwise refinement, functional testing, structural testing), various designtechnologies (function decomposition, object oriented design), various life cycle models (waterfall,Cleanroom) by running controlled experiments and case studies to determine the effectiveness of eachof these technologies in the environment. Based upon the proposed models, we evaluated and feedbackinformation to the project managers. This allowed them to better estimate the resources required,predict the kinds of defects that take place in their environment, and understand which aspects of theirdevelopment process that were at risk. We formally developed the Quality Improvement Paradigm andbegan formalizing process, product, and quality models for the environment. This allowed us to tailorwhat we learned about the effectiveness of reading and testing for the SEL environment and change thebasic processes based upon the results of prior projects. Naturally the GQM and various other modelscontinued to evolve.During this phase we better understood the experimental nature of software development and how toapply the Quality Improvement Paradigm to the resulting processes. This entailed defining and refiningprocess, product, knowledge and quality models.The standard SEL FORTRAN development model8.3was more precisely defined. For example, changes were made to take advantage