Slide 1Transport in DatacentersData Center Packet TransportTCP in the Data CenterDatacenters and OLDIsPartition/Aggregate Application StructureGenerality of Partition/AggregateWorkloadsImpairmentsIncastQueue BuildupData Center Transport RequirementsTension Between RequirementsData Center TransportSlide 15Review: The TCP/ECN Control LoopTwo Key IdeasData Center TCP AlgorithmDCTCP in ActionWhy it WorksEvaluationCluster Traffic BenchmarkBaselineBaselineBaselineBaselineSlide 27D2TCP’s Main Idea and ContributionsD2TCP Cont’dD2TCP: Congestion AvoidanceD2TCP: Gamma Correction FunctionGamma Correction Function (cont.)Gamma Correction Function (cont.)Gamma Correction Function (cont.)D2TCP: Stability and ConvergenceD2TCP: Deadline-aware SchedulingWorkloadsMissed DeadlinesPerformance of Long-lived FlowsSlide 40Transport in DatacenterspFabric in 1 SlideConceptual modelDC Fabric: Just a Giant SwitchDC Fabric: Just a Giant SwitchDC Fabric: Just a Giant SwitchSlide 47“Ideal” Flow Schedulingpfabric designKey InsightpFabric SwitchpFabric Switch ComplexitypFabric Rate ControlpFabric Rate ControlWhy does this work?Overall Average FCTMice FCT (<100KB)ConclusionConclusion (DCTCP)Conclusion (D2TCP)Conclusion (pFabric)Slide 62Richard AntiabongUniversity of Texas at DallasTransport Layer Protocols for Data Centers1Most of the Slides adapted from the original conference presentationsTransport in Datacenters21000s of server portsDC network interconnect for distributed compute workloadsMsg latency is King traditional “fairness” metrics less relevant webwebappappdbdbmap-reducemap-reduceHPCHPCmonitoringmonitoringcachecacheData Center Packet Transport•Large purpose-built DCs–Huge investment: R&D, business•Transport inside the DC–TCP rules (99.9% of traffic)•How’s TCP doing?3TCP in the Data Center•We’ll see TCP does not meet demands of apps.–Suffers from bursty packet drops, Incast [SIGCOMM ‘09], ... –Builds up large queues: Adds significant latency. Wastes precious buffers, esp. bad with shallow-buffered switches.•Operators work around TCP problems.‒Ad-hoc, inefficient, often expensive solutions‒No solid understanding of consequences, tradeoffs4Datacenters and OLDIsOLDI = OnLine Data Intensive applicationse.g., Web search, retail, advertisementsAn important class of datacenter applicationsVital to many Internet companiesOLDIs are critical datacenter applicationsTLAMLAMLAWorker Nodes………Partition/Aggregate Application Structure6Picasso“Everything you can imagine is real.” “Bad artists copy. Good artists steal.”“It is your work in life that is the ultimate seduction.““The chief enemy of creativity is good sense.““Inspiration does exist, but it must find you working.”“I'd like to live as a poor man with lots of money.““Art is a lie that makes us realize the truth.“Computers are useless. They can only give you answers.”1.2.3. …..1. Art is a lie…2. The chief… 3. …..1.2. Art is a lie… 3. …..Art is…Picasso• Time is money Strict deadlines (SLAs)• Missed deadline Lower quality resultDeadline = 250msDeadline = 50msDeadline = 10msGenerality of Partition/Aggregate•The foundation for many large-scale web applications.–Web search, Social network composition, Ad selection, etc.•Example: FacebookPartition/Aggregate ~ Multiget–Aggregators: Web Servers–Workers: Memcached Servers7Memcached ServersInternetWebServersMemcached ProtocolWorkloads•Partition/Aggregate (Query)•Short messages [50KB-1MB] (Coordination, Control state)•Large flows [1MB-50MB] (Data update) 8Delay-sensitiveDelay-sensitiveThroughput-sensitiveImpairments•Incast•Queue Buildup•Buffer Pressure9Incast10TCP timeoutWorker 1Worker 2Worker 3Worker 4AggregatorRTOmin = 300 ms• Synchronized mice collide. Caused by Partition/Aggregate.Queue Buildup11Sender 1Sender 2Receiver• Big flows buildup queues. Increased latency for short flows.• Measurements in Bing cluster For 90% packets: RTT < 1ms For 10% packets: 1ms < RTT < 15msData Center Transport Requirements121. High Burst Tolerance–Incast due to Partition/Aggregate is common.2. Low Latency–Short flows, queries3. High Throughput –Continuous data updates, large file transfersThe challenge is to achieve these three together.Tension Between Requirements13High Burst ToleranceHigh ThroughputLow LatencyDCTCPDeep Buffers: Queuing Delays Increase LatencyShallow Buffers: Bad for Bursts & Throughput Reduced RTOmin (SIGCOMM ‘09) Doesn’t Help LatencyAQM – RED: Avg Queue Not Fast Enough for IncastObjective:Low Queue Occupancy & High ThroughputData Center TransportTwo techniques currently exist for meeting requirements of Data center transport.Traditional TCP approachData Center TCP (DCTCP)-Non-deadline awareDeadline-Aware Data Center TCP (D2TCP)-deadline awareNetwork assisted approachD3 Preemptive Distributed Quick (PDQ)pFabric14The DCTCP Algorithm15Review: The TCP/ECN Control Loop16Sender 1Sender 2ReceiverECN Mark (1 bit)ECN = Explicit Congestion NotificationTwo Key Ideas1. React in proportion to the extent of congestion, not its presence.Reduces variance in sending rates, lowering queuing requirements.2. Mark based on instantaneous queue length.Fast feedback to better deal with bursts.18ECN Marks TCP DCTCP1 0 1 1 1 1 0 1 1 1 Cut window by 50% Cut window by 40%0 0 0 0 0 0 0 0 0 1 Cut window by 50% Cut window by 5%Data Center TCP AlgorithmSwitch side:– Mark packets when Queue Length > K.19Sender side:–Maintain running average of fraction of packets marked (α).In each RTT:Adaptive window decreases:–Note: decrease factor between 1 and 2.BKMarkDon’t MarkDCTCP in Action20Setup: Win 7, Broadcom 1Gbps SwitchScenario: 2 long-lived flows, K = 30KB(Kbytes)Why it Works1. High Burst ToleranceLarge buffer headroom → bursts fit.Aggressive marking → sources react before packets are dropped.2. Low LatencySmall buffer occupancies → low queuing delay.3. High Throughput ECN averaging → smooth rate adjustments, low variance.21Evaluation•Implemented in Windows stack. •Real hardware, 1Gbps and 10Gbps experiments–90 server testbed–Broadcom Triumph 48 1G ports – 4MB shared memory–Cisco Cat4948 48 1G ports – 16MB shared memory–Broadcom Scorpion 24 10G ports – 4MB shared memory•Numerous micro-benchmarks– Throughput and Queue
View Full Document
Unlocking...