1 Analyzing the Evolution of Large-Scale Software Systems using Design Structure Matrices and Design Rule Theory: Two Exploratory Cases Matthew J. LaMantia VMware, Inc. [email protected] Yuanfang Cai Drexel University [email protected] Alan D. MacCormack, John Rusnak Harvard Business School [email protected] [email protected] Abstract Designers have long recognized the value of modularity, but important software modularity principles have remained informal. According to Baldwin and Clark’s [1] design rule theory (DRT) , modular architectures add value to system designs by creating options to improve the system by substituting or experimenting on individual modules. In this paper, we examine the design evolution of two software product platforms through the modeling lens of DRT and design structure matrices (DSMs). We show that DSM models and DRT precisely explain how real-world modularization activities in one case allowed for different rates of evolution in different software modules and in another case conferred distinct strategic advantages on a firm by permitting substitution of an at-risk software module without substantial change to the rest of the system. Our results provide positive evidence that DSM and DRT can inform important aspects of large-scale software structure and evolution, having the potential to guide software architecture design activities. 1. Introduction Designers have long recognized the value of modularity. Constantine’s low-coupling, high-cohesion principle has been well known since the 1970’s [17]. Parnas’s information hiding criterion [12] has remained influential for decades. Designers are educated to seek modular architectures to better accommodate expected changes and to enable parallel development. However, because these principles are informal, their successful application depends on intuition and experience. Intuition and experience, in turn, do not prevent a big company from constantly grappling with unanticipated dependencies, modularity decay, and delays in bringing software to market. The significant delay of Windows Vista exemplifies the case: Even a firm with deep expertise in software development, like Microsoft, can still suffer from a complexity disaster resulting from a system’s lack of modularity [6]. Thus we are in need of a formal theory and models of modularity and software evolution that can capture the essence of these important but informal design principles and provide the power of description, prediction and prescription. In recent work, Baldwin and Clark [1] propose a theory to explain how modular architectures add value to system designs by creating options to improve the system by substituting or experimenting on individual modules. Their theory explains the relationship between the modular structure of the IBM 360 computer and its subsequent successful evolution, but it has not been applied to large-scale software systems. Baldwin and Clark’s theory, which is based on Steward’s [17] design structure matrix (DSM) modeling approach (described below), argues that design rules can be used to resolve interdependencies and create modular architectures by specifying the interface between modules. Sullivan et al. [18] applied this approach to Parnas’s [13] small but canonical Key Word in Context (KWIC) design example. They show that DSM models and design rule theory can precisely capture Parnas’s information hiding criterion. To further explore the theory’s descriptive power for large and complex software systems, we examine the evolution of two software product platforms through the lens of DSM models and design rule theory: (1) Tomcat, an open source web application server from the Apache Software Foundation; and (2) a proprietary application server from a company which remains anonymous. Both systems have been evolving for years. Their designers have refactored the systems several times and released multiple versions. Our case studies show that a theory based on DSMs, design rules and options precisely explains how modularization conferred strategic advantage on firms by allowing codebases to evolve in particular ways. They provide positive evidence that the model and theory have the power to formally explain phenomena2 related to the evolution of large-scale software systems. The designers of the systems we examine made their decisions based on their visions and prior experiences. We find that design rule theory and DSM models help to formalize their previously informal intuitions and judgments, while also revealing parts of the system that may need to be further modularized. This paper is organized as follows: Section 2 introduces DSM modeling and Baldwin and Clark’s design rule theory. Section 3 presents the methodology of our case study and the research questions we aim to answer. Section 4 presents the case study of Tomcat. Section 5 presents the case study of the proprietary product platform. Section 6 discusses the implications of our results. Section 7 presents our ongoing and future work. Section 8 describes related work, and Section 9 concludes. 2. DSM Modeling and Design Rule Theory This section introduces DSM modeling and explains how design rules decouple otherwise coupled design decisions, create options, and enable independent substitution. In the rest of the paper, we will refer to the formal analysis of design rules and options as “design rule theory” (DRT). The design structure matrix (DSM) was initially conceived by Steward [17], and later developed by Eppinger et al. [5] as means of modeling interactions between design variables of engineered systems. A DSM is a square matrix, in which each design variable corresponds both to a row and a column of the matrix. A cell is checked if and only if the design decision corresponding to its row depends on the design decision corresponding to the column. A DSM represents modules as blocks along the diagonal. The left DSM in Figure 1 shows a simple DSM with three modules. A and B model software design decisions that depend on each other, for example, two procedures that call each other, two classes that refer to each other, or a data structure and algorithm that go hand in hand. The designers usually need to break the cyclical dependency so that A and B can be changed without affecting each other. In contrast, the dependency between C and A models a layered architecture. C refers to A, but A does not refer to C. Building