CLEANROOM PROCESS MODEL The philosophy behind Cleanroom software engineering is to ovoid dependence on costly defect-removal processes by writing code increments right the first time and verifying their correctness before testing. its process model incorporates the statistical quality certification of code increments as they accumulate into a system. RICHARD C. LINGER IBM Cleanroom Software Technology Center T oday’s competitive pressures and societv’s increasing denendence on I Y I software have led to a new focus on devel- opment processes. The Cleanroom pro- cess, which has evolved over the last de- cade, has demonstrated that it can improve both the productivity of developers who use it and the quality of the software they produce. Cleanroom software engineering is a team-oriented process that makes devel- opment more manageable and predictable This article is based on a paper that appears in R-w. 1% Id C’mf .?%$wure E&g.., f!ZEE CS Press, Las Alamitos, Calif., 1993, pp. 2.13.The IEEESoftwareEditorial Board has selected it as the best practice paper presented at ICSE-15. The bard thanks Richard A DeMillo, ICSE- I5 Program Cochair, for his help in arranging for its pub- lication in IEEE Sofim. because it is done under statistical quality control. Cleanroom is a modern approach to software development. In traditional, craft-based development, defects are re- garded as inevitable and elaborate defect- removal techniques are a part of the devel- opment process. In such a process, software proceeds from development to unit testing and debugging, then to func- tion and system testing for more debug- ging. In the absence of workable altema- tives, managers encourage programmers to get code into execution quickly, so de- b ugging can begin. Today, developers rec- ognize that defect removal is an error- prone, inefficient activity that consumes resources better allocated to getting the code right the first time. 50 07407459/94/5M co 0 1994 IEEE MARCH 1994Cleanroom teams at IBM and other organizations are achieving remarkable quality results in both new-system devel- opment and modifications and extensions to legacy systems. The quality of software produced by Cleanroom development teams is sufficient (often near zero defects) for the software to enter system testing directly for first-ever execution by test teams. The theoretical foundations of Clean- room - formal specification and design, correctness verification, and statistical testing - have been reduced to practice and demonstrated in nearly a million lines of code. Some Cleanroom projects are profiled in the box on p. 56. QUALITY COMPARISON Quality comparisons between tradi- tional methods and the Cleanroom pro- cess are meaningful when measured from first execution. Most traditional develop- ment methods begin to measure errors at function testing (or later), omitting errors found in private unit testing. A traditional project experiencing, say, five errors per thousand lines of code (KLOC) in func- non testing may have encountered 2S or more errors per KLOC when measured from first execution in unit testing. At entry to unit testing, traditional soft- ware typically exhibits 25 to 35 or more errors per KLOC.’ In contrast, the weighted average of errors found in 17 Cleanroom projects, involving nearly a million lines of code, is 2.3 errors per KLOC. This number represents all errors found in all testing, measured from first- ever execution through test completion - it is the average number of residual errors present after the development team has performed correctness verification. In addition to this remarkable differ- ence in the number of errors, experience has shown a qualitative difference in the complexity of errors found in Cleanroom versus traditional software. Errors left be- hind by Cleanroom correcmess verifica- tion tend not to be complex design or in- terface errors, but simple mistakes easily found and fixed by statistical testing. In this article, I describe the Clean- room development process, from specifi- cation and design through correctness verification and statistical usage testing for quality certification. INCREMENTAL DEVELOPMENT The Cleanroom process is based on developing and certifying a pipeline of software increments that accumulate into the final system. The increments are de- veloped and certified by small, indepen- dent teams, with teams of teams for large projects. System integration is continual, and func;ionality g&s with the addition of successive increments. In this ap- proach, the harmonious operation of future incre- ments at the next level of rehnement is predehned by increments already in execution, thereby mini- mizing interface and de- sign errors and helping developers maintain in- tellectual control. The Cleanroom de- velopment process is in- CLEANROOM DEVELOPMENT IS INTENDED TO BE “QUICK AND CLEAN,” NOT “QUICK AND DII As the figure shows, Cleanroom devel- opment involves two cooperating teams and five major activities: + Specification. Cleanroom develop- ment begins with specification. Together, the development team and the certifica- tion team produce two specifications: functional and usage. Large projects may have a separate specification team. The functional specification defines the required external system behavior in all circumstances of use; the usage specifi- cation defines usage scenarios and their probabilities for all possible system usage, both correct and incorrect. The func- tional specification is the RN.” tended to be “quick and clean,” not “quick and dirty.” The idea is to quickly develop the right product with high quality for the user, then go on to the next version to incorporate new requirements arising from user experience. In the Cleanroom process, correcmess is built in by the development team through formal specification, design, and verification. Team correctness verification takes the place of unit testing and