System of Systems Lead System Integrators: Where do They Spend Their Time and What Makes Them More/Less Efficient? ******************************** Background for COSOSIMO by Jo Ann Lane [email protected] University of Southern California Center for Software Engineering June 2005  USC CSE 2005SoS LSIs: Where Do They Spend Their Time Page 2 June 2005  USC CSE 2005 ABSTRACT As organizations strive to expand system capabilities through the development of system-of-systems (SoS) architectures, they want to know "how much effort" and "how long". In order to answer these questions, it is important to first understand the types of activities performed in SoS architecture development and integration and how these vary across different SoS implementations. This paper provides preliminary results of research conducted to determine types of SoS Lead System Integrator (LSI) activities and how these differ from the more traditional system engineering activities described in EIA 632 (Processes for Engineering a System). It also looks at concepts in organizational theory, complex adaptive systems, and chaos theory and how these might be applied to SoS LSI activities to improve success rates and efficiency in the development of these “very large” complex systems. Introduction As organizations strive to expand system capabilities through the development of system-of-systems (SoS) architectures, they want to know "how much effort" and "how long". Efforts are currently underway at the University of Southern California (USC) Center for Software Engineering (CSE) to develop a cost model to estimate the effort associated with SoS Lead System Integrator (LSI) activities. The research described in this paper is in support of the development of this cost model, COSOSIMO. Research conducted to date in this area has focused more on technical characteristics of the SoS. Feedback from USC CSE industry affiliates [7] indicates that the extreme complexity typically associated with SoS architectures and political issues between participating organizations have a major impact on the LSI effort. This is also supported by surveys of system acquisition managers [1] and studies of failed programs [11]. The focus of this directed research project is to further investigate effort and schedule issues on “very large” SoS programs such as Future Combat Systems (FCS) and to determine organizational characteristics that significantly impact overall success and productivity of the program. Scope Because the USC CSE Industry affiliates and other surveys and studies have indicated that organizational and political issues can significantly impact the success of very large, complex system development efforts as well as the productivity and total effort required to accomplish the system development objectives, we continue to investigate related theories, studies, and surveys in this area. The specific goals of the research effort presented in this paper are to: 1. More clearly define the LSI activities for which the model will estimate effort and show how they differ from the more traditional system engineering activities described in EIA 632 [4]. 2. Identify organizational factors from organizational, complexity, and chaos theory that may apply to LSI activities.SoS LSIs: Where Do They Spend Their Time Page 3 June 2005  USC CSE 2005 3. Analyze actual effort data from several LSI teams with different organizational characteristics, and through this analysis, identify sets of LSI activities that are impacted by each factor identified in (2) above. As indicated above, initial plans for this research included the collection and analysis of effort data from sample SoS activities. However, due to contract delays, this data was not available for this research effort. Therefore, this paper only presents the results of 1 and 2 above. Background The first step in developing a cost model is to describe the effort you are trying to estimate as well as the characteristics of the activities that you are trying to estimate. Key to the COSOSIMO cost estimation model is understanding what an SoS is, how it is different from the more traditional system, and what are the major sources of effort. What is an SoS: So to begin, we started with a basic definition of an SoS. An initial survey of the literature shows that there are a considerable number of definitions of SoSs. It also became clear during this survey that it would be extremely difficult, if not impossible to develop a single cost model that could handle all of the various types of SoSs. Therefore, we looked at defining a set of SoSs that would benefit from a cost model [10] and tailoring the development of COSOSIMO to that set of SoSs. After reviewing the various SoS concepts, the authors concluded that the SoSs of interest to the cost model development effort had the following characteristics: • SoSs are very-large systems developed by creating a framework or architecture to integrate existing systems, systems currently under development, as well as new systems still to be developed. Many of the existing SoS system components were initially developed as a closed, stovepipe type of system that cannot easily share information with other systems. • There are SoS sponsoring organizations and user communities that have a long-term vision at some level of what the SoS should be, are responsible for the development and maintenance of the SoS, and are interested in development cost information and tradeoffs based on SoS characteristics and development approaches. (This is in opposition to business systems that are integrated over time with no real long-term vision or strategy, but rather based on current fiscal funding and new technologies on the near-term horizon.) • SoS development activities are planned and coordinated by an LSI organization. • SoS system components are typically systems that can operate on their own outside of the SoS framework. They are also independently developed and managed throughout their full lifecycle by organizations other than the LSI. As shown in Figure 1, most SoS architectures are described as net-centric architectures that allow the system components to exchange information and perform tasks within the framework that they are not capable of performing on their own outside of the framework. This is often referred to as an “emergent behavior”. Key issues in developing an SoS are the