Slide 1context: massive systemselectricity expenseswhat is being doneour proposalexploiting price volatilityexploiting price volatilitysystem model (status quo)request routing frameworkwill our proposal work?will our proposal work?will our proposal work?will our proposal work?will our proposal work?how much can we save by exploiting price volatility?generality of resultsrequest routing evaluationrequest routing schemeAkamai workloadelectricity pricesrequest routing evaluationlocation energy modelimportance of elasticitybandwidth costsbandwidth constraintslatency constraintspractical implicationsconclusionSlide 29market diversityAsfandyar Qureshi (MIT)Rick Weber (Akamai)Hari Balakrishnan (MIT)John Guttag (MIT)Bruce Maggs (Duke/Akamai)cutting the electric bill for internet-scale systemsÉole @ flickr2context: massive systemsQureshi • SIGCOMM • August 2009 • Barcelona • SpainGoogle:estimated maptens of locations in the US>0.5M serversmajor data centermajor data centerothersthousands of servers / multiple locationsAmazon, Yahoo!, Microsoft, AkamaiBank of America (≈50 locations), Reuters3electricity expensesmillions spent annually on electricityGoogle ~ 500k custom servers ~ $40 million/yearAkamai ~ 40k off-the-rack servers ~ $10 million/yearelectricity costs are growingsystems are rapidly increasing in sizeoutpacing energy efficiency gainsrelative cost of electricity is rising3-year server total cost of ownership by 2012: ›electricity ≈ 2 × hardwarebandwidth prices are fallingQureshi • SIGCOMM • August 2009 • Barcelona • Spain4what is being donereduce number of kWhenergy efficient hardwarevirtualization and consolidationpower off servers when possiblecooling (air economizers instead of chillers, etc.)dc power distribution, etc.reduce cost per kWhbuild data-centers where average price is lowQureshi • SIGCOMM • August 2009 • Barcelona • Spain5our proposalexploit electricity market dynamicsgeographically uncorrelated price volatilitymonitor real-time market prices and adapt request routingskew load across clusters based on pricesleverage service replication and spare capacityadapting to real-time prices is a new idea…complementary to energy efficiency workQureshi • SIGCOMM • August 2009 • Barcelona • Spain6exploiting price volatilityQureshi • SIGCOMM • August 2009 • Barcelona • Spain0255075100VirginiaVirginiaCaliforniaCaliforniaIllinoisIllinoisRT market price $/MWhtime (hours)day one day two day threelocational pricing not well correlated CA-VA correlation ≈ 0.2locational pricing not well correlated CA-VA correlation ≈ 0.2hourly variation peaks ~ $350/MWh negative priceshourly variation peaks ~ $350/MWh negative prices3 of the largest data center markets3 of the largest data center markets7exploiting price volatilityQureshi • SIGCOMM • August 2009 • Barcelona • Spain0255075100CaliforniaCaliforniaRT market price $/MWhtime (hours)day one day two day threeVirginiaVirginiaCalifornia has min. priceCalifornia has min. priceVirginia has min. priceVirginia has min. price8system model (status quo)Qureshi • SIGCOMM • August 2009 • Barcelona • SpainCaliforniaCaliforniaVirginiaVirginiaIllinoisIllinoissystem9electricity prices (hourly)electricity prices (hourly)request routing frameworkQureshi • SIGCOMM • August 2009 • Barcelona • Spainperformanceaware routingperformanceaware routingrequestsbandwidth price modelbandwidth price modelnetwork topologynetwork topologylatency goalslatency goalscapacity constraintscapacity constraintsbest-price performance aware routingbest-price performance aware routingmap:requests to locationsmap:requests to locationswill our proposal work?will our proposal work?does electricity usage depend on server load?how much can we reduce a location’s electricity consumption by routing clients away from it?will our proposal work?does electricity usage depend on server load?latency concernshow far away from a client is the cheap energy?will our proposal work?does electricity usage depend on server load?latency concernsbandwidth costs could risecheaper electricity ~ more expensive bandwidth?will our proposal work?does electricity usage depend on server load?latency concernsbandwidth costs could riseis there enough spare capacity?how much can we save by exploiting price volatility? today: large companies more than $1M/year with better technology: more than $10M/year better than placing all servers in cheapest market16generality of resultsAkamai-specific inputsclient workloadgeographic server distribution (25 cities / non-uniform)capacity & bandwidth constraintsresults should apply to other systemsrealistic client workload›2000 content providers›hundreds of billions of requests per dayrealistic server distribution›better than speculating…Qureshi • SIGCOMM • August 2009 • Barcelona • Spain17electricity prices (hourly)electricity prices (hourly)request routing evaluationQureshi • SIGCOMM • August 2009 • Barcelona • Spainperformanceaware routingperformanceaware routingrequestsbandwidth price modelbandwidth price modelnetwork topologynetwork topologylatency goalslatency goalscapacity constraintscapacity constraintsbest-price performance aware routingbest-price performance aware routingmap:requests to locationsmap:requests to locations18request routing schemeperformance-aware price optimizermap client -> set of locations that meets latency goalsrank locations based on electricity pricesremove locations nearing capacity from setpick top-ranked locationassumptionscomplete replicationhourly route updates preserve stabilityuniform bandwidth prices (we will relax this later…)Qureshi • SIGCOMM • August 2009 • Barcelona • Spain19Akamai workloadmeasured traffic on Akamai’s CDNlarge subset of Akamai’s servers (~20K) in 25 citiescollected over 24 days (Dec 2008 – Jan 2009)5-min samples›number of hits and bytes transferred›track how Akamai routed clients to clusters›group clients by origin statealso derived a synthetic workloadQureshi • SIGCOMM • August 2009 • Barcelona • Spain20electricity pricesextensive survey of US electricity marketsregional wholesale markets (both futures and
View Full Document
Unlocking...