SEDA: An Architecture for Well-Conditioned,Scalable Internet ServicesMatt Welsh, David Culler, and Eric BrewerComputer Science DivisionUniversity of California, Berkeley{mdw,culler,brewer}@cs.berkeley.eduAbstractWe propose a new design for highly concurrent Internet services, whichwe call the staged event-driven architecture (SEDA). SEDA is intendedto support massive concurrency demands and simplify the construc-tion of well-conditioned services. In SEDA, applications consist of anetwork of event-driven stages connected by explicit queues. This ar-chitecture allows services to be well-conditioned to load, preventingresources from being overcommitted when demand exceeds service ca-pacity. SEDA makes use of a set of dynamic resource controllers tokeep stages within their operating regime despite large fluctuations inload. We describe several control mechanisms for automatic tuningand load conditioning, including thread pool sizing, event batching, andadaptive load shedding. We present the SEDA design and an implemen-tation of an Internet services platform based on this architecture. Weevaluate the use of SEDA through two applications: a high-performanceHTTP server and a packet router for the Gnutella peer-to-peer file shar-ing network. These results show that SEDA applications exhibit higherperformance than traditional service designs, and are robust to hugevariations in load.1 IntroductionThe Internet presents a computer systems problem of unprecedentedscale: that of supporting millions of users demanding access to servicesthat must be responsive, robust, and always available. The number ofconcurrent sessions and hits per day to Internet sites translates into aneven higher number of I/O and network requests, placing enormousdemands on underlying resources. Yahoo! receives over 1.2 billionpage views daily [62], and AOL’s Web caches service over 10 billionhits a day [2]. Moreover, Internet services experience huge variations inservice load, with bursts coinciding with the times that the service hasthe most value. The well-documented “Slashdot Effect” shows that it isnot uncommon to experience more than 100-fold increases in demandwhen a site becomes popular [58]. As the demand for Internet servicesgrows, new system design techniques must be used to manage this load.For more information as well as a source-code re-lease of the software described in this paper, please seehttp://www.cs.berkeley.edu/˜mdw/proj/seda/.This systems challenge is magnified by three trends that increasethe generality of services. First, services themselves are becomingmore complex, with static content replaced by dynamic content thatinvolves extensive computation and I/O. Second, service logic tends tochange rapidly, which increases the complexity of engineering and de-ployment. Third, services are increasingly hosted on general-purposefacilities, rather than on platforms that are carefully engineered for aparticular service. As these trends continue, we envision that a richarray of novel services will be authored and pushed into the infrastruc-ture where they may become successful enough to scale to millionsof users. Several investigations are addressing the high-level aspectsof service authorship, including naming, lookup, composition, and ver-sioning [16, 21, 22, 53, 55]. We focus here on the performance aspect ofthe problem: achieving robust performance on a wide range of servicessubject to huge variations in load, while preserving ease of authorship.Replication is a key aspect of service scalability. Given a serviceinstance that can sustain a certain level of performance, it must be repli-cated to sustain a many-fold increase in load. Scalable clusters are nowwidely used to obtain replication within a service site [18], and wide-area replication is increasingly employed for specific services, such ascontent distribution networks [1, 3, 19]. However, because the peakload may be orders of magnitude greater than the average, it is notpractical to replicate most services to handle the maximum potentialdemand. Therefore, we expect large spikes in the load experienced byeach node. Our goal is to develop a general framework for authoringhighly concurrent and well-conditioned service instances that handleload gracefully.Unfortunately, traditional operating system designs and widely pro-moted models of concurrency do not provide this graceful managementof load. Commodity operating systems focus on providing maximaltransparency by giving each process the abstraction of a virtual machinewith its own CPU, memory, disk, and network. This goal is somewhat atodds with the needs of Internet services, which demand massive concur-rency and extensive control over resource usage. Processes and threadsare well-supported models of concurrent programming, but often entailhigh overhead in terms of context-switch time and memory footprint,which limits concurrency. Transparent resource virtualization preventsapplications from making informed decisions, which are vital to man-aging excessive load.Much work has focused on performance and robustness for specificservices [4, 24, 44, 63]. However, with services becoming increas-ingly dynamic and flexible, this engineering burden becomes excessive.Few tools exist that aid the development of highly concurrent, well-conditioned services; our goal is to reduce this complexity by providinggeneral-purpose mechanisms that aid software developers in obtainingthese properties.We propose a new design framework for highly concurrent serverapplications, which we call the staged event-driven architecture(SEDA).1SEDA combines aspects of threads and event-based program-ming models to manage the concurrency, I/O, scheduling, and resourcemanagement needs of Internet services. In SEDA, applications are con-structed as a network of stages, each with an associated incoming eventqueue. Each stage represents a robust building block that may be indi-vidually conditioned to load by thresholding or filtering its event queue.In addition, making event queues explicit allows applications to makeinformed scheduling and resource-management decisions, such as re-ordering, filtering, or aggregation of requests. SEDA makes use of dy-namic resource throttling to control the resource allocation and schedul-ing of application components, allowing the system to adapt to overloadconditions.This paper describes the design, architecture, and implementationof a SEDA-based Internet services platform. This platform