1 Introduction2 Developing and Using Feasibility Evidence for Life Cycle Commitment Milestones3 Nature of FEDs and AP Milestones4 Problems Encountered without a FED5 Problems Avoidable with FED6 AT&T Experience with AP Reviews7 ICM Phases, Anchor Points, and Decision Options8 Key Point: Need to Show Evidence9 Off-Nominal Architecture-Breakers10 Common Examples of Inadequate Evidence11 Examples of Making the Evidence Adequate12 ICM Detailed Activity Descriptions13 Focus of Each Commitment Review14 Lean Risk Management Plans14.1 Objectives14.2 Deliverables and Milestones14.3 Responsibilities14.4 Approach14.5 Resources15 FED Development Process Framework16 Steps for Developing Feasibility Evidence17 Commitment Review Process Overview18 Example of FED Risk Evaluation Criteria19 Large-Scale Simulation and Testbed FED Preparation Example20 Conclusions21 References7th Annual Conference on Systems Engineering Research 2009 (CSER 2009) Better Management of Development Risks: EarlyFeasibility EvidenceBarry Boehm1 and Jo Ann Lane2 1 University of Southern California, USA, [email protected] University of Southern California, USA, [email protected] The Incremental Commitment Model (ICM) organizes more rapid and thorough concurrent systems engineering and acquisitionprocesses in ways that provide points at which they can synchronize and stabilize, and at which their risks of going forward canbe better assessed and fitted into a risk-driven stakeholder resource commitment process. In particular, its concurrent activitiesof Understanding Needs, Envisioning Opportunities, System Scoping and Architecting, Feasibility Evidence Development, andRisk/Opportunity Assessment enable projects to focus specifically on their system constraints and environments and onopportunities to deal with them. This paper describes in detail the content, preparation, and role of feasibility evidence at keydecision points in the system development cycle and how this can be used to effectively identify and manage risks throughout thedevelopment cycle. The feasibility evidence is not intended to assess just a single sequentially developed system definitionelement, but rather to assess the consistency, compatibility, and feasibility of several concurrently-engineered elements. To makethis concurrency work, a set of anchor point milestone reviews are performed to ensure that the many concurrent activities aresynchronized, stabilized, and risk-assessed at the end of each phase using developer-produced, expert-validated feasibilityevidence.Keywords – anchor point milestones, feasibility evidence, Incremental Commitment Model 1 IntroductionThe Incremental Commitment Model (ICM) organizesmore rapid and thorough concurrent systems engineeringand acquisition processes in ways that provide points atwhich they can synchronize and stabilize, and at which theirrisks of going forward can be better assessed and fitted intoa risk-driven stakeholder resource commitment process. Inparticular, its concurrent activities of Understanding Needs,Envisioning Opportunities, System Scoping andArchitecting, Feasibility Evidence Development, andRisk/Opportunity Assessment enable projects to focusspecifically on their system constraints and environmentsand on opportunities to deal with them.This paper describes in detail the content, preparation, androle of feasibility evidence at key decision points in thesystem development cycle and how this can be used toeffectively identify and manage risks throughout thedevelopment cycle. The feasibility evidence is not intendedto assess just a single sequentially developed systemdefinition element, but rather to assess the consistency,compatibility, and feasibility of several concurrently-engineered elements. To make this concurrency work, a setof anchor point milestone reviews are performed to ensurethat the many concurrent activities are synchronized,stabilized, and risk-assessed at the end of each phase. Eachof these anchor point milestone reviews is focused ondeveloper-produced and expert-validated evidence,documented in a Feasibility Evidence Description (FED)document, to help the key stakeholders determine whetherto proceed into the next level of commitment. The FED is based on evidence from simulations, models, orexperiments with planned technologies and increasinglydetailed analysis of development approaches and projectedproductivity rates. The parameters used in the analysesshould be based on measured component performance or onhistorical data showing relevant past performance, costestimation accuracy, and actual developer productivityrates. The FED is not an optional appendix, but a first-classdeliverable subject to earned value management andindependent expert review. It has been used effectivelyfrom small web-services projects to ultra-large systems ofsystems [1,2,3,4,5].This paper also provides experience-based types ofevidence that should be evaluated at each commitmentreview, criteria for evaluating the evidence, and alternativecourses of action based upon the results of the evaluation:risk level is acceptable—proceed ahead; number or level ofrisks are too high and need further mitigation beforeproceeding—do more work; or risk is unacceptable—rescope or discontinue program. 2 Developing and Using Feasibility Evidence for Life Cycle Commitment MilestonesSuccess-Critical Stakeholders (SCSHs) for systems underdevelopment have the authority and the responsibility toexercise accountable governance of their organization’sLoughborough University – 20th - 23rd April 20097th Annual Conference on Systems Engineering Research 2009 (CSER 2009) resources when committing such resources to the nextphase of system development. In many cases, when usingprocesses other than the ICM, these stakeholders arepresented with insufficient information upon which to makea responsible decision.As a result, many projects proceed into system developmentwith little idea of whether their project requirements,architectures, plans, budgets, and schedules are feasible andconsistent or not. This is one of the major reasons for