Cluster-Based Scalable Network Services Armando Fox Steven D. Gribble Yatin Chawathe Eric A. Brewer Paul Gauthier University of California at Berkeley (fox, gribble, yatin, biewer} @cs.berkeley.edu We identifit three fundamental requirements for scalable net- work services: incremental scalability and oveflow growth provi- sioning, 24x7 availability through fault masking, and cost- effectiveness. We argue that clusters of commodity workstations interconnected by a high-speed SAN are exceptionally well-suited to meeting these challenges for Internet-server workloads, pro- vided the software infrastructure for managing partial failures and administering a large cluster does not have to be reinvented for each new service. To this end, we propose a general, layered archi- tecture for building cluster-based scalable network services that encapsulates the above requirements for reuse, and a service-pro- gramming model based on composable workers thatpe$onn trans- formation, aggregation, caching, i and customization (TACC) of Internet content. For both performance and implementation sim- plicity, the architecture and TACC programming model exploit BASE, a weaker-than-ACID data semantics that results from trad- ing consistency for availability and relying on sof state for robust- ness in failure management., Our ,qychitecture can be used as an “off the shelf * irtfrastructural platfonn for creating new network services, allowing authors to focus on the “content” of the service {by composing TACC building blocks) rather than its implementa- tion. We discuss two real implemeritations of services based on this architecture: TranSend, a Web distillation proxy deployed to the UC Berkeley dialup IP population, and HotBot, the commercial implementation of the Inktomi search engine. We present detailed measurements of TranSend’s performance based on substantial cli- ent traces, as well as anecdotal evidence from the TranSend and HotBot experience, to support the claims made for the architecture. 1 Introduction “One of the overall design goals is to create a computing system which is capable of meeting almost all of the requirements of a large computer utility. Such systems must run continuously and reliably 7 days a week, 24 hours a day... and must be capable of meeting wide service demands. “Because the system must ultimately be comprehensive and able to aa’apt to unknown future requirements, its framework must be general, and cbpable of evolving over time.” - Corbat6 and Vyssotsky on Multics, 1965 [I71 Although it is normally viewed as an operating system, Multics (Multiplexed Information and Computer Service) was originally conceived as an infrastructural computing service, so it is not sur- prising that its goals as stated above are similar to our own. The primary obstacle to deploying Multics was the absence of the net- work infrastructure, which is now in place. Network applications have exploded in popularity in part because they are easier to man- age and evolve than their desktop application counterparts: they eliminate the need for software distribution, and simplify customer service and bug tracking by avoiding the difficulty of dealing with multiple platforms and versions. Also, basic queueing theory permission to make digital/hard copy of part or all this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advan- tage, the copyright notice, the title of the publication and its date appear, and notice is given that copying is by permission of ACM, Inc. To copy otherwise, to republish, to post on servers. or to redistribute to lists, requires prior specific permission and/or a fee. SOSp-16 IO/97 Saint-Malo, France @ 1997 ACM 0-89791-916-5/97/0010...$3.50 Inktomi Corporation { gauthier} @inktomi.com shows that a large central ‘(virtual) server is more efficient in both cost and utilization than a collection of smaller servers; standalone desktop systems represent the degenerate case of one “server” per user. All of these support the argument for Network Computers Pa However, network services remain difficult to deploy because of three fundamental challenges: scalability, availability and cost effectiveness. l By scalabiliry, we mean that when the load offered to the service increases, an incremental and linear increase in hardware can maintain the same per-user level of service. l By availability, we mean that the service as a whole must bc available 24x7, despite transient partial hardware or softwaro failures. l By cost effectiveness, we mean that the service must bc economical to administer and expand, even though it potentially comprises many workstation nodes. We observe that clusters of workstations have some fundnmen- tal properties that can be exploited to meet these requirements: using commodity PCs as the unit of scaling allows the service to ride the leading edge of the cost/performance curve, the inheront redundancy of clusters can be used to mask transient failuros, and “embarrassingly parallel” network service workloads map well onto networks of workstations. However, developing cluster soft- ware and administering a running cluster remain complex. The pri- mary contributions of this work are the design, analysis, and implementation of a layered framework for building network ser- vices that addresses this complexity. New services can use thls framework as an off-the-shelf solution to scalability, availability, and several other problems, and focus instead on the content of the service being developed. The lower layer handles scalability, avall- ability, load balancing, support for bursty offered load, and system monitoring and visualization, while the middle layer provldcs extensible support for caching, transformation among MIME types, aggregation of information from multiple sources, and pcr- sonalization of the service for each of a large number of users (mass customization). The top layer