Slide 1AgendaData Center CostsServer CostsGoal: Agility – Any service, Any ServerAchieving AgilityNetwork ObjectivesAgendaThe Network of a Modern Data CenterSlide 10No Performance IsolationSlide 12Slide 13What Do Data Center Faults Look Like?AgendaAgendaSwitch on Chip ASICsPackagingLatencyAgendaMeasuring Traffic in Today’s Data CentersFlow CharacteristicsTraffic Matrix VolatilityToday, Computation Constrained by Network*Congestion: Hits Hard When it Hits*AgendaVL2: Distinguishing Design PrinciplesWhat Enables a New Solution Now?An Example VL2 Topology: Clos NetworkUse Randomization to Cope with VolatilitySlide 31Embracing End SystemsVL2 PrototypeVL2 Achieves Uniform High ThroughputVL2 Provides Performance IsolationVL2 is resilient to link failuresSummarySlide 381Albert Greenberg, ICDCS 2009 keynoteNetworking The CloudAlbert GreenbergPrincipal [email protected](work with James Hamilton, Srikanth Kandula, Dave Maltz, Parveen Patel, Sudipta Sengupta, Changhoon Kim)2Albert Greenberg, ICDCS 2009 keynoteAlbert Greenberg, ICDCS 2009 keynoteAgenda•Data Center Costs–Importance of Agility•Today’s Data Center Network•A Better Way?3Albert Greenberg, ICDCS 2009 keynoteData Center Costs•Total cost varies–Upwards of $1/4 B for mega data center–Server costs dominate–Network costs significantAmortized Cost* Component Sub-Components~45% Servers CPU, memory, disk~25% Power infrastructure UPS, cooling, power distribution~15% Power draw Electrical utility costs~15% Network Switches, links, transit*3 yr amortization for servers, 15 yr for infrastructure; 5% cost of moneyThe Cost of a Cloud: Research Problems in Data Center Networks. Sigcomm CCR 2009. Greenberg, Hamilton, Maltz, Patel.4Server CostsUgly secret: 30% utilization considered “good” in data centersCauses include:•Uneven application fit:–Each server has CPU, memory, disk: most applications exhaust one resource, stranding the others•Long provisioning timescales:–New servers purchased quarterly at best•Uncertainty in demand:–Demand for a new service can spike quickly•Risk management:–Not having spare servers to meet demand brings failure just when success is at hand•Session state and storage constraints–If the world were stateless servers, life would be good5Goal: Agility – Any service, Any Server•Turn the servers into a single large fungible pool–Let services “breathe” : dynamically expand and contract their footprint as needed•Benefits–Increase service developer productivity–Lower cost–Achieve high performance and reliabilityThe 3 motivators of most infrastructure projects6Achieving Agility•Workload management–Means for rapidly installing a service’s code on a server–Virtual machines, disk images •Storage Management–Means for a server to access persistent data–Distributed filesystems (e.g., blob stores) •Network–Means for communicating with other servers, regardless of where they are in the data center7Network ObjectivesDevelopers want a mental model where all their servers, and only their servers, are plugged into an Ethernet switch1. Uniform high capacity–Capacity between servers limited only by their NICs–No need to consider topology when adding servers2. Performance isolation–Traffic of one service should be unaffected by others3. Layer-2 semantics–Flat addressing, so any server can have any IP address–Server configuration is the same as in a LAN–Legacy applications depending on broadcast must work8Agenda•Data Center Costs–Importance of Agility•Today’s Data Center Network•A Better Way?9Albert Greenberg, ICDCS 2009 keynoteAlbert Greenberg, ICDCS 2009 keynoteThe Network of a Modern Data Center•Hierarchical network; 1+1 redundancy•Equipment higher in the hierarchy handles more traffic, more expensive, more efforts made at availability scale-up design•Servers connect via 1 Gbps UTP to Top of Rack switches•Other links are mix of 1G, 10G; fiber, copperRef: Data Center: Load Balancing Data Center Services, Cisco 2004InternetInternetCR CRAR AR AR AR…SSLB LBData CenterLayer 3InternetSS…SS……Layer 2Key:• CR = L3 Core Router• AR = L3 Access Router• S = L2 Switch• LB = Load Balancer• A = Rack of 20 servers with Top of Rack switch~ 4,000 servers/pod10Albert Greenberg, ICDCS 2009 keynoteAlbert Greenberg, ICDCS 2009 keynoteInternal Fragmentation Prevents Applications from Dynamically Growing/Shrinking•VLANs used to isolate properties from each other•IP addresses topologically determined by ARs•Reconfiguration of IPs and VLAN trunks painful, error-prone, slow, often manualInternetInternetCR CR…AR ARSSLB LBSSAAA…SSAAA……AR ARSSLB LBSSAAA…SSAAA…A11Albert Greenberg, ICDCS 2009 keynoteAlbert Greenberg, ICDCS 2009 keynoteNo Performance Isolation•VLANs typically provide reachability isolation only•One service sending/receiving too much traffic hurts all services sharing its subtreeInternetInternetCR CR…AR ARSSLB LBSSAAA…SSAAA……AR ARSSLB LBSSAAA…SSAAA…ACollateral damage12Albert Greenberg, ICDCS 2009 keynoteAlbert Greenberg, ICDCS 2009 keynoteNetwork has Limited Server-to-Server Capacity, and Requires Traffic Engineering to Use What It Has•Data centers run two kinds of applications:–Outward facing (serving web pages to users)–Internal computation (computing search index – think HPC)InternetInternetCR CR…AR ARSSLB LBSSAAA…SSAAA……AR ARSSLB LBSSAAA…SSAAA…10:1 over-subscription or worse (80:1, 240:1)13Albert Greenberg, ICDCS 2009 keynoteAlbert Greenberg, ICDCS 2009 keynoteNetwork Needs Greater Bisection BW, and Requires Traffic Engineering to Use What It Has•Data centers run two kinds of applications:–Outward facing (serving web pages to users)–Internal computation (computing search index – think HPC)InternetInternetCR CR…AR ARSSLB LBSSAAA…SSAAA……AR ARSSLB LBSSAAA…SSAAA…Dynamic reassignment of servers and Map/Reduce-style computations mean traffic matrix is constantly changingExplicit traffic engineering is a nightmare14Albert Greenberg, ICDCS 2009 keynoteAlbert Greenberg, ICDCS 2009 keynoteWhat Do Data Center Faults Look Like?•Need very high reliability near top of the tree–Very hard to achieve Example: failure of a temporarily unpaired core switch affected ten million users for four hours–0.3% of failure events knocked out all members of a network redundancy groupRef: Data Center: Load Balancing Data Center Services, Cisco
View Full Document
Unlocking...