16 IEEE SOFTWARE January/February 2001 0740-7459/00/$10.00 © 2001 IEEEWhat is software quality assurance,and where is it headed? Has it be-come passé as firms embrace rapidapplication development tech-niques, modern tools, and spiraldevelopment models? These are in-teresting questions to ask in light of SQA’shistory, quality models, and technologicaldevelopments.There is, of course, the old question: is SQAa discipline or an organization? MatthewFisher and I discussed this issuemost recently in a chapter in theHandbook of Software QualityAssurance.1Past and current prac-tices contradict its status as a disci-pline; object-oriented design is adiscipline, but it is difficult to castSQA in the same light. The factalone that many different organi-zations practice SQA in many dif-ferent ways makes it difficult tocharacterize it as a discipline. Discipline or organization?If we look at organizations in the commer-cial world, especially those in the informationsystems arena, SQA consists of testing—pri-marily, system testing. In many cases, due topoor project planning or project management,little time is available to test adequately. Doc-umentation of the requirements is often un-available, so the test program’s ability to de-tect software defects is suspect. Testing asSQA is like locking the barn door after thehorse has escaped; it is hardly “assuring prod-uct quality,” nor is it the discipline of SQA.Members of organizations that haveadopted the Capability Maturity Model(CMM) often view SQA as the “process po-lice.” In this role, SQA determines whether thedevelopers (and project management) conformto the process policies, standards, and proce-dures. Work products are checked for compli-ance with templates defining their content andformat. In these organizations, those organiza-tions that have a separate SQA function mighthave people with little software backgroundperforming SQA. Deviations from the processmight not be as apparent to them as to moreknowledgeable SQA personnel.The product assurance department Imanaged several years back did a number ofthings. It defined the organization’s process,it checked for compliance with it, evaluatedwork products for content quality and con-formance to format and content require-ments, moderated inspections, and did sometesting. Many other companies practicedSQA this way, as well—very differentlyfrom those I described earlier.Today, a set of practices that tend to varyquite a bit emerges, hardly fitting the de-scription of a discipline. SQA is not alwaysan organization. Nor, when it is an organiza-tion, is it necessarily an independent one—acharacteristic necessary to ensure objectiveevaluations. In many organizations, develop-ers time-share their activities. They spendpart of the time doing development workand part doing whatever has been defined asSQA for that organization. Clearly, there isambiguity on that score, as well.Technological developmentsWhat effect does technology have on thepractice of SQA? In an article on softwaremanagement, Walker Royce cites assessingquality with an independent team and ex-haustive inspection as principles of conven-managerWhich Way, SQA?Emanuel R. BakerEditor: Donald J. Reifer ■ Reifer Consultants ■ [email protected]/February 2001IEEE SOFTWARE 17MANAGERtional software management.2Thisapproach is an outgrowth, he as-serts, of the waterfall model of soft-ware development. Using differentapproaches, such as the spiral modeland other iterative development ap-proaches, lets us use moremodern software manage-ment techniques. Roycestates, Modern software develop-ment produces the archi-tecture first, followed byusable increments of par-tial capability, and thencompleteness. Require-ments and design flawsare detected and resolvedearlier in the life cycle,avoiding the big-bang inte-gration at the end of aproject. Quality controlimproves because systemcharacteristics inherent inthe architecture (such asperformance, fault tolerance, inter-operability, and maintainability)are identifiable earlier in theprocess where problems can becorrected without jeopardizing tar-get costs and schedules.He later argues that the conven-tional principle of a separate qualityassurance results in “projects that iso-late ‘quality police,’” and that “a bet-ter approach is to work quality as-sessment into every activity throughthe checks and balances of organiza-tional teams focused on architecture,components, and usability.” Another fault of conventional soft-ware management, he suggests, is slav-ish insistence on requirements trace-ability to design. Clearly, one of SQA’sobjectives is to ensure that the designand resultant code implements theagreed-upon requirements. The prob-lem is that demanding rigorous problem-to-solution traceability is frequentlycounterproductive, forcing thedesign to be structured in thesame manner as the requirements.Good component-based archi-tectures have chaotic traceabilityto their requirements. Tight prob-lem-to-solution traceability mighthave been productive when100% custom software was thenorm; those days are gone.The QA role that emerges from thesediscussions is one of evaluationsembedded in the development pro-cess. The process Royce describes de-emphasizes separate SQA and pro-cess policing, rightly viewing them ascounterproductive.E-commerce SQAIt’s interesting to look at Royce’sapproach in contrast to developmentpractices followed by e-commercesoftware houses. We often hearabout doing things “at Internetspeed.” In e-commerce (and the prac-tices of many shrink-wrap softwarehouses), that often involves gettingapplications out the door as quicklyas possible to beat the competition.Process is of little concern to many e-commerce organizations. Few e-com-merce firms have demonstrated anyinterest in adopting quality modelssuch as the CMM. (For that matter,few shrink-wrap software houseshave, either.) Consequently, the con-sumer often gets flawed software. For a long time, liability was notan issue; the consumer merely putup with flawed software. However,with the advent of the Internet ande-commerce, liability is more of aconcern. An e-commerce applica-tion’s failure to provide adequateprotection against credit card num-ber theft, for example, raises seri-ous liability issues. Various studies indicatethat 30% to 40% of soft-ware projects fail, that up to50% of the failures are dueto inadequate requirementsdefinition, and that up to40% of the failures