Richard Stutzke- - Estimating Project Risk Reserves-v7 Estimating Project Risk Reserves Richard D. Stutzke Science Applications International Corp. 6725 Odyssey Drive Huntsville. AL 35806 (256) 971-6624 (ofc) (256) 971-6550 (fax) (256) 971-7330 (asst) 23 October 2002 Presented at the International Forum on COCOMO and Software Cost Modeling, 22-25 October 2003, Los Angeles, CA. ABSTRACT All projects have risks. To provide resources needed to avert or mitigate these risks, planners must estimate a risk roserve. Setting a "reasonable " reserve is important. A large reserve may cause the project's bid price to be noncompetitive. A small reserve may be inadequate to handle the problems that will inevitably occur. This paper describes a method to estimate an "optimal" reserve for a list of identified risks. For each risk, the estimator provides the probability and consequences (cost) of occurrence before mitigation, the cost of actions that could be performed to mitigate the risk, and the probability and consequences that would remain after performing these actions. The Risk Reduction Leverage, RRL, is the difference in the expected losses before and after mitigation, divided by the cost of mitigation. (RRL is similar to a Return On Investment except it uses expected costs instead of total estimated costs.) The method ranks the risks based on their Risk Reduction Leverage, and applies criteria to decide which risks should be mitigated and which should be accepted. The method also gives criteria for deciding which risks should be mitigated immediately ("preventative ") and which can be deferred ("contingent "). The paper suggests ways to handle overlapped risks and mitigation actions, coupled risks, and compounded consequences. This paper describes a spreadsheet that implements the method, and provides an example showing how the reserve is calculated. (This is a hard reserve because it is based on documented calculations that are tied to specific risks.) It explains how to use the data for planning, and for tracking the identified risks during a project. This paper is based on Chapter 17, Estimating the Impacts of Risks, in "Software and System Estimation: Project, Products, and Process " by R. D. Stutzke, to appear in 2003. 02002 by Richard D. Stutzke 1 9/30/2002 10:43 AMRichard Stutzke--Estimating Project Risk Reserves-v7 1.0 Introduction Risks are inevitable for any complicated project due to imperfect knowledge about the product's requirements and operating conditions, the technology and tools being used, and staff capabilities, experience, and continuity. Ignorance of various factors compounds the problem. In addition, systems are developed in an environment that constantly changes, making these factors difficult to predict. A risk is an uncertain condition or event that, if it occurs, could have a negative effect on the project's cost and schedule, or on the products it produces. Risks have a probability of occurrence and an associated cost of occurrence. (Uncertain events that occur and provide benefits to the project are called opportunities.) The product of the probability of occurrence and the cost of occurrence is called the impact. (Some authors call this quantity the risk exposure.) Project planners, assisted by engineers, managers, and other experts, identify risks that could jeopardize project success. These individuals also identi@ possible mitigation actions that can be performed to reduce the impact of each identified risk. A typical mitigation action for hardware is over engineering the item to provide a large safety factor. (This is calculated based on the known strength of materials and the configuration of the component parts. The engineers design a structure that will support some multiple of the maximum expected load. The multiple is the safety factor.) Safety factors are commonly used in constructing bridges and buildings. Hardware engineers can also mitig~te the risk of system failure by designing secondary backup systems that can contain or limit the damage, or can perform essential functions if the primary system fails. For example, elevators have brakes that prevent the elevator from falling if the cable breaks. For software systems there is no analogy to the strength of materials, nor are there easy ways to provide backup systems. One approach that has been discussed in the literature is multi-version programming [Leveson, 19951. Unfortunately, adding backup functions and the logic needed to invoke them (switchover logic) actually increases the amount of software and its associated complexity. Increasing software size and complexity is known to ncrease the total number of latent defects in the software, and so increases the probability of system failure. Another way of addressing many kinds of risk is to define Terms and Conditions in the contract or in the product's written warranty. The provider of the product or service states in the contract the provider is not liable for certain types of failures that may occur. ("It is not my fault.") In some cases, the producer may purchase insurance to cover the occurrence of losses. To mitigate risks requires resources. The challenge faced by project planners and estimators is how large the reserve should be. Estimating a value that is too small means that the project will lack the necessary funds to perform the mitigation actions or to cover the costs associated with the damage due to failures. On the other hand, choosing an amount that is too large means that the bid price will be excessively high, resulting in loss 02002 by Richard D. StutzkeRichard Stutzke--Estimating Project Risk Reserves-v7 Stutzk of the bid. Another concern in many organizations is that the reserve be quantitatively justified. This is called a hard reserve. In contrast, a soft reserve is simply in monetary amount that is asserted without any quantitative justification. Planners must also decide how to expend the reserves that are provided. Some risks have a very