164 Performance Evaluation of Software Architectures Lloyd G. Williams Software Engineering Research 264 Ridgeview Lane Boulder, Colorado 80302 (303) 938-9847 [email protected] Abstract There is growing recognition of the importance of the role of architecture in determining the quality attributes, such as modifiability, reusability, reliability, and performance, of a software system. While a good architecture cannot guarantee attainment of quality goals, a poor architecture can prevent their achievement. This paper discusses assessment of the performance characteristics of software architectures. We describe the information required to perform such assessments and discuss how it can be extracted from architectural descriptions. The process of evaluating the performance characteristics of a software architecture is described and illustrated with a simple case study. 1. Introduction There is growing recognition of the role of architecture in determining the quality of a software system [23], [14], [12]. While decisions made at every phase of the development process can impact the quality of software, architectural decisions have the greatest impact on quality attributes such as modifiability, reusability, reliability, and performance. As Clements and Northrop note: “Whether or not a system will be able to exhibit its desired (or required) quality attributes is largely determined by the time the archttecture is chosen.” [I21 While a good architecture cannot guarantee attainment of quality goals, a poor architecture can prevent their achievement. Since architectural decisions are among the earliest made in a software development project and can have the greatest impact on software quality, it is important to support assessment of quality attributes at the time these decisions are made. Our work focuses on early assessment of software architectures to ensure that they will meet non- Connie U. Smith Performance Engineering Services PO Box 2640 Santa Fe, New Mexico, 87504-2640 (505) 988-38 11 http://www.perfeng.com/-cusmith functional, as well as functional, requirements. For this paper, we focus on the assessment of the performance characteristics of a software architecture since many software systems fail to meet performance objectives when they are initially implemented. Performance is an important quality attribute of software systems. Performance failures result in damaged customer relations, lost productivity for users, lost revenue, cost overruns due to tuning or redesign, and missed market windows. Moreover, “tuning” code to improve performance is likely to disrupt the original architecture, negating many of the benefits for which the architecture was selected. Finally, it is unlikely that “tuned” code will ever equal the performance of code that has been engineered for performance. In the worst case, it will be impossible to meet performance goals by tuning, necessitating a complete redesign or even cancellation of the project. Our experience is that most performance failures are due to a lack of consideration of performance issues early in the development process, in the architectural design phase. Poor performance is more often the result of problems in the architecture rather than in the implementation. As Clements points out: “Performance is largely a function of the frequency and nature of inter-component communication, in addition to the performance characteristics of the components themselves, and hence can be predicted by studying the architecture of a system.” [I I] In this paper we describe the use of software performance engineering (SPE) techniques to perform early assessment of a software architecture to determine whether it will meet performance objectives. The use of SPE at the architectural design phase can help developers select a suitable architecture. Continued application of SPE techniques throughout the development process helps insure that performance goals are met. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee”. WOSP96, Santa Fe, NM. (c) 1996 ACM l-58113-060-O/98/00... $5.00Our previous work extended Software Performance Engineering (SPE) methods to include specific techniques for evaluating the performance of object-oriented systems. That work focused on early life-cycle issues and introduced Use Cases as the bridge between object- oriented methods and SPE. However, it did not specifically address issues related to larger architectural concerns. This paper extends our previous work to include extensions to the SPE methods that address the use of SPE for making architectural tradeoff decisions. The next section contrasts related work. Section 3 provides the SPE process steps appropriate for performance evaluations of software architectures. Section 4 explains the sources of information for the SPE evaluations, then section 5 illustrates the process with a simple case study. The last section presents a summary and conclusions. 2. Related Work Kazman and co-workers have proposed a scenario-based approach to the analysis of software architectures [21], [20]. Their approach considers various stakeholders in the system (e.g., users, system administrators, maintainers) and develops usage scenarios from their various points of view. These scenarios are expressed informally as one sentence descriptions. They typically capture uses of the system that are related to quality attributes, such as ease of modification. The architecture is then evaluated on how much work is required to satisfy the scenarios.