Optimal Power Allocation in Server FarmsAnshul GandhiCarnegie Mellon UniversityPittsburgh, PA, [email protected] Harchol-Balter∗Carnegie Mellon UniversityPittsburgh, PA, [email protected] DasIBM ResearchHawthorne, NY, [email protected] LefurgyIBM ResearchAustin, TX, [email protected] farms today consume more than 1.5% of the totalelectricity in the U.S. at a cost of nearly $4.5 billion. Giventhe rising cost of energy, many industries are now seekingsolutions for how to best make use of their available power.An important question which arises in this context is howto distribute available power among servers in a server farmso as to get maximum performance.By giving more power to a server, one can get higher serverfrequency (speed). Hence it is commonly bel ieved that, fora given power budget, performance can be maximized byoperating servers at their highest power levels. However,it is also conceivable that one might prefer to run serversat their lowest power levels, which allows more servers to beturned on for a given power budget. To fully understand theeffect of power allocation on performance in a server farmwith a fixed power budget, we introduce a queueing theo-retic model, which allows us to predict the optimal powerallocation in a variety of scenarios. Results are verified viaextensive experiments on an IBM BladeCenter.We find that the optimal power allocation varies for differ-ent scenarios. In particular, it is not always optimal to runservers at their maximum power levels. There are scenar-ios where it might be optimal to run servers at their lowestpower levels or at some intermediate power levels. Our anal-ysis shows that the optimal power allocation is non-obviousand depends on many factors such as the power-to-frequencyrelationship in the processors, the arrival rate of jobs, themaximum server frequency, the lowest attainable server fre-quency and the server farm configuration. Furthermore, ourtheoretical model allows us to explore more general settingsthan we can implement, including arbitrarily large serverfarms and different power-to-frequency curves. Importantly,we show that the optimal power allocation can significantly∗Research supported by NSF SMA/PDOS Grant CCR-0615262 and a 2009 IBM Faculty Award.Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.SIGMETRICS/Performance’09, June 15–19, 2009, Seattle, WA, USA.Copyright 2009 ACM 978-1-60558-511-6/09/06 ...$5.00.improve server farm performance, by a factor of typically1.4 and as much as a factor of 5 in some cases.Categories and Subject DescriptorsC.4 [Performance of Systems]: Modeling techniquesGeneral TermsTheory, Experimentation, Measurement, Performance1. INTRODUCTIONServers today consume ten times m ore power than theydid ten years ago [3, 21]. Recent articles estimate that a300W high performance server requires more than $330 ofenergy cost per year [24]. Given the large number of serversin use today, the worldwide expenditure on enterprise powerand cooling of these servers is estim ated to be in excess of$30 billion [21].Power consumption is particularly pronounced in CPUintensive server farms composed of tens to thousands ofservers, all sharing workload and power supply. We considerserver farms where each incoming job can be routed to anyserver, i. e. , it has no affinity for a particular server.Server farms usually have a fixed peak power budget. Thisis because large power consumers operating server farms areoften billed by power suppliers, in part, based on their peakpower requirements. The peak power budget of a serverfarm also determines its cooling and power delivery infras-tructure costs. Hence, companies are interested in maxi-mizing the performance at a server farm given a fixed peakpower budget [4, 8, 9, 21].The power allocation problem we consider is: how to dis-tribute available power among servers in a server farm soas to minimize mean response time. Every server running agiven workload has a minimum level of power consumption,b, needed to operate the processor at the lowest allowablefrequency and a maximum level of power consumption, c,needed to operate the processor at the highest allowable fre-quency. By varying the power allocated to a server withinthe range of b to c Watts, one can proportionately vary theserver frequency (See Fig. 2). Hence, one might expect thatrunning servers at their highest power levels of c Watts,which we refer to as PowMax, is the optimal power allocationscheme to minimize response time. Since we are constrainedby a p ower budget, there are only a limited number of serversthat we can operate at the highest power level. The rest ofthe servers remain turned off. Thus PowMax corresponds tohaving few fast servers. In sharp contrast is PowMin, whichwe define as operating servers at their lowest power levels ofb Watts. Since we spend less power on each server, PowMincorresponds to having many slow servers. Of course theremight b e scenarios where we neither operate our servers atthe highest power levels nor at the lowest power levels, butwe operate them at some intermediate power levels. Werefer to such power allocation schemes as PowMed.Understanding power allocation in a server farm is intrin-sically difficult for many reasons: First, there is no singleallocation scheme which is optimal in all scenarios. For ex-ample, it is commonly believed that PowMax is the optimalpower allocation scheme [1, 7]. However, as we show later,PowMin and PowMed can sometimes outperform PowMaxby almost a factor of 1.5. Second, it turns out that theoptimal power allocation depends on a very long list of ex-ternal factors, such as the outside arrival rate, whether anopen or closed workload configuration is used, the power-to-frequency relationship (how power translates to serverfrequency) inherent in the technology, the minimum powerconsumption of a server (b Watts), the maximum power thata server can use (c Watts), and many other factors. It is sim-ply impossible to examine all these factors via experiments.To fully understand the effect of power allocation on meanresponse time in a