Building Security InEditor: Gary McGraw, [email protected] trend, most systems for designingsoftware also tend to describe posi-tive features. Savvy software practitioners arebeginning to think beyond features,touching on emergent properties ofsoftware systems such as reliability, se-curity, and performance. This ismostly because experienced cus-tomers are beginning to demand se-cure and reliable software; but inmany situations, it’s still up to the soft-ware developer to define “secure”and “reliable.” To create secure and reliable soft-ware, we first must anticipate abnor-mal behavior. We don’t normally de-scribe non-normative behavior in usecases, nor do we describe it withUML, but we must have some way totalk about and prepare for it. “Mis-use” (or “abuse”) cases can help orga-nizations begin to see their software inthe same light that attackers do. Bythinking beyond normative features,while simultaneously contemplatingnegative or unexpected events, soft-ware security professionals can betterunderstand how to create secure andreliable software. Guttorm Sindre and AndreasOpdahl extend use-case diagramswith misuse cases to represent the ac-tions that systems should prevent intandem with those that they shouldsupport for security and privacy re-quirement analysis.1Ian Alexanderadvocates using misuse and use casestogether to conduct threat and hazardanalysis during requirements analy-sis.2In this article, we provide a non-academic introduction to the soft-ware security best practice of misuseand abuse cases, showing you how toput the basic science to work. In caseyou’re keeping track, Figure 1 showsyou where we are in our series of arti-cles about software security’s place inthe software development life cycle.Security is not a set of featuresThere is no convenient security pull-down menu that will let you select“security” and then sit back andwatch magic things happen. Unfor-tunately, many software developerssimply link functional security fea-tures and mechanisms somewhereinto their software, mistakenly assum-ing that doing so addresses securityneeds throughout the system. Toooften, product literature makes broad,feature-based claims about security,such as “built with SSL” or “128-bitencryption included,” which repre-sent the vendor’s entire approach forsecuring its product.Security is an emergent propertyof a system, not a feature. This is likehow “being dry” is an emergent prop-erty of being inside a tent in the rain.The tent only keeps you dry if thepoles are stabilized, vertical, and ableto support the weight of wet fabric;the tent also must have waterprooffabric (with no holes) and be largeenough to protect everyone whowants to remain dry. Lastly, everyonemust remain under the tent the entiretime it’s raining. So, although havingpoles and fabric is important, it’s notenough to say, “the tent has poles andfabric, thus it keeps you dry!” This sortof claim, however, is analogous to theclaims that software vendors makewhen they highlight numbers of bitsin crypto keys or the use of particularencryption algorithms. Cryptographyof one kind or another is usually nec-essary to create a secure system, but se-curity features alone are not sufficientfor building secure software.Because security is not a feature,it can’t be bolted on after other soft-ware features are codified, nor can itbe patched in after attacks have oc-curred in the field. Instead, it must bebuilt in from the ground up, as a crit-ical part of the design from the verybeginning (requirements specifica-tion) and included in every subse-quent development phase all the waythrough fielding a complete system. Sometimes building security in atthe beginning means making explicittrade-offs when specifying system re-quirements. For example, ease of usemight be paramount in a medical sys-tem designed for secretaries in doc-tors’ offices, but complex authentica-tion procedures, such as obtaining andusing a cryptographic identity, can behard to use.3Furthermore, regulatorypressures from HIPAA and Califor-nia’s new privacy regulations (SB1386) force designers to negotiate areasonable trade-off. PACO HOPEAND GARYMCGRAWCigitalANNIE I.ANTO´NNorthCarolina StateUniversitySoftware development is all about making softwaredo something: when software vendors sell theirproducts, they talk about what the products do tomake customers’ lives easier, such as encapsulatingbusiness processes or something similarly positive. Following thisMisuse and Abuse Cases:Getting Past the Positive32 PUBLISHED BY THE IEEE COMPUTER SOCIETY ■ 1540-7993/04/$20.00 © 2004 IEEE ■ IEEE SECURITY & PRIVACYBuilding Security InTechnical approaches must go farbeyond the obvious features, deepinto the many-tiered heart of a soft-ware system to provide enough secu-rity: authentication and authorizationcan’t stop at a program’s front door.The best, most cost-effective ap-proach to software security incorpo-rates thinking beyond normative fea-tures and incorporates that thinkingthroughout the development process.Every time a new requirement, fea-ture, or use case is created, someoneshould spend some time thinkingabout how that feature might be un-intentionally misused or intentionallyabused. Professionals who know howfeatures are attacked and how to pro-tect software should play an active rolein this kind of analysis.Thinking about what you can’t doAttackers are not standard-issue cus-tomers. They’re bad people withmalicious intentions who want yoursoftware to act to their benefit. If thedevelopment process doesn’t addressunexpected or abnormal behavior,then an attacker usually has plenty ofraw material with which to work.4Attackers are creative, but despitethis, they’ll always probe well-knownlocations—boundary conditions,edges, intersystem communication,and system assumptions—in thecourse of their attacks. Clever attack-ers will try to undermine the assump-tions on which a system was built. If adesign assumes that connections fromthe Web server to the database serverare always valid, for example, an at-tacker will try to make the Web serversend inappropriate requests to accessvaluable data. If the software design as-sumes that the client never modifies itsWeb browser cookies before they aresent back to the requesting server (inan attempt to preserve some state), at-tackers will intentionally cause prob-lems by modifying the cookies. Build-ing Secure Software teaches us that wehave