Internet Server ClustersUsing Clusters for Scalable Services[Fox/Brewer]: SNS, TACC, and All ThatTACCTranSend StructureSNS/TACC PhilosophyTACC Examples(Worker) Ignorance Is BlissQuestionsSNS/TACC Functional IssuesPorcupine: A Highly Available Cluster-based Mail ServiceWhy Email?GoalsConventional Mail SolutionKey Techniques and RelationshipsPorcupine ArchitecturePorcupine OperationsBasic Data StructuresPorcupine AdvantagesAvailabilitySoft-state ReconstructionHow does Porcupine React to Configuration Changes?Hard-state ReplicationMore on Porcupine ReplicationHow Efficient is Replication?Slide 26Load balancing: Deciding where to store messagesSlide 28Clusters: A Broader ViewWolfpack: ResourcesFault-Tolerant Systems: The Big PictureWolfpack: Resource Placement and MigrationMembership 101Failure DetectorsFailure Detectors in Real SystemsMembership ServiceExample: WombatWombat BasicsNode Arrival/Recovery in WombatNode Failure in WombatLeader Failure in WombatMultiple Failures in WombatSuppressing False LeadersPartitions in WombatHealing a PartitionWombat: WrinklesInternet Server ClustersInternet Server ClustersJeff ChaseDuke University, Department of Computer ScienceCPS 212: Distributed Information SystemsUsing Clusters for Scalable ServicesUsing Clusters for Scalable ServicesClusters are a common vehicle for improving scalability and availability at a single service site in the network.Are network services the “Killer App” for clusters?•incremental scalabilityjust wheel in another box...•excellent price/performancehigh-end PCs are commodities: high-volume, low margins•fault-tolerance“simply a matter of software”•high-speed cluster interconnects are on the marketSANs + Gigabit Ethernet...cluster nodes can coordinate to serve requests w/ low latency•“shared nothing”[Fox/Brewer]: SNS, TACC, and All That[Fox/Brewer]: SNS, TACC, and All That[Fox/Brewer97] proposes a cluster-based reusable software infrastructure for scalable network services (“SNS”), such as:•TranSend: scalable, active proxy middleware for the Webthink of it as a dial-up ISP in a box, in use at Berkeleydistills/transforms pages based on user request profiles•Inktomi/HotBot search enginecore technology for Inktomi Inc., today with $15B market cap. “bringing parallel computing technology to the Internet”Potential services are based on Transformation, Aggregation, Caching, and Customization (TACC), built above SNS.TACCTACCVision: deliver “the content you want” by viewing HTML content as a dynamic, mutable medium.1. Transform Internet content according to:•network and client needs/limitationse.g., on-the-fly compression/distillation [ASPLOS96], packaging Web pages for PalmPilots, encryption, etc.•directed by user profile database2. Aggregate content from different back-end services or resources.3. Cache content to reduce cost/latency of delivery.4. Customize (see Transform)TranSend StructureTranSend Structure$$$FrontEndsProfilesControlPanelhtml gif jpgTo InternetSAN (high speed)Utility (10baseT)Coordination bus$Cache partition...Datatype-specific distiller[adapted from Armando Fox (through http://ninja.cs.berkeley.edu/pubs)]SNS/TACC PhilosophySNS/TACC Philosophy1. Specify services by plugging generic programs into the TACC framework, and compose them as needed.sort of like CGI with pipesrun by long-lived worker processes that serve request queuesallows multiple languages, etc. 2. Worker processes in the TACC framework are loosely coordinated, independent, and stateless.ACID vs. BASEserve independent requests from multiple usersnarrow view of a “service”: one-shot readonly requests, and stale data is OK3. Handle bursts with designated overflow pool of machines.TACC ExamplesTACC ExamplesHotBot search engine•Query crawler’s DB•Cache recent searches•Customize UI/presentationTranSend transformation proxy•On-the-fly lossy compression of inline images (GIF, JPG, etc.)•Cache original & transformed•User specifies aggressiveness, “refinement” UI, etc.CTT$$AATT$$CDBDBhtmlhtml[Fox](Worker) Ignorance Is Bliss(Worker) Ignorance Is BlissWhat workers don’t need to know•Data sources/sinks•User customization (key/value pairs)•Access to cache•Communication with other workers by nameCommon case: stateless workersC, Perl, Java supported•Recompilation often unnecessary•Useful tasks possible in <10 lines of (buggy) Perl[Fox]QuestionsQuestions1. What are the research contributions of the paper?system architecture decouples SNS concerns from contentTACC programming model composes stateless worker modulesvalidation using two real services, with measurements How is this different from clusters for parallel computing?2. How is this different from clusters for parallel computing?3. What are the barriers to scale in SNS/TACC?4. How are requests distributed to caches, FEs, workers?5. What can we learn from the quantitative results?6. What about services that allow client requests to update shared data?e.g., message boards, calendars, mail,SNS/TACC Functional IssuesSNS/TACC Functional Issues1. What about fault-tolerance?•Service restrictions allow simple, low-cost mechanisms.Primary/backup process replication is not necessary with BASE model and stateless workers.•Uses a process-peer approach to restart failed processes.Processes monitor each other’s health and restart if necessary.Workers and manager find each other with “beacons” on well-known ports.2. Load balancing?•Manager gathers load info and distributes to front-ends.•How are incoming requests distributed to front-ends?Porcupine: A Highly Available Cluster-Porcupine: A Highly Available Cluster-based Mail Servicebased Mail ServiceYasushi SaitoBrian BershadHank LevyUniversity of Washington Department of Computer Science and Engineering,Seattle, WAhttp://porcupine.cs.washington.edu/[Saito]Why Email?Why Email?Mail is importantReal demandMail is hardWrite intensiveLow localityMail is easyWell-defined APILarge parallelismWeak consistency [Saito]How much of Porcupine isreusable to other services?Can we use the SNS/TACCframework for this?GoalsGoalsUse commodity hardware to build a large, scalable mail serviceThree facets of scalability ...Performance: Linear increase with cluster size Manageability: React to changes automaticallyAvailability: Survive failures gracefully[Saito]Conventional Mail SolutionConventional Mail SolutionStatic partitioningPerformance problems:No dynamic load balancingManageability