Xen-ophobia: On profiling boot startupTheophilus A. Benson, Steven Kappestbenson, [email protected] Sciences DepartmentUniversity of Wisconsin, MadisonMay 15, 2008AbstractThe fact that systems fail should come as no sur-prise to anymore who has ever developed or workedon a system. A failure reduces the availabil-ity of the system and hence the productivity ofentities using this system. To increase systemavailability, several approaches have been de-veloped; these approaches range from simpletechniques s uch as restarting the entire system,to complex algorithms that isolate and restartthe failed subsystem. We find the simple ap-proach of a system reboot to be particularly in-teresting because it is a widely d eployed ap-proach. We profile the set of instructions ex-cuted during the startup p h ase of the sys temand identify heavily utilized segments. We im-plement, in Xen, a framework for both monitor-ing the start-up sequence and identifying highlyutilized segmen ts of code. We show that mod-ifications t o the identified segments affect thestart-up sequence. Finally, we examine the iden-tified segments and suggest modifications thatwill, if implemented, increase availability by re-ducing the boot up time.1 IntroductionThe system restart p lays a key role in t h e recov-ery model for many large and popular syst ems.For example, Th e B lue Screen of Death or sys-tem failure, is perceived as the default recoverybehavior for failures in the Windows O.S. [4]. Inthe famous Blue Screen of Death scenario, Win-dows panics and prompts the user t o restart th esystem. System reboot as a recovery model isnot native to Windows Operating Systems, in[7], P r abhakaran et al. show that the sy stem re-boot model is employed by se ve r al Linux filesystems. There are many reasons for the preva-lent us age of this model. The reasons rangefrom simplicity of implementation t o negligibleoverhead. Although simple, this model is notwithout its drawbacks; it forces the sys tem toperform the entire boot-up sequence. The boot-up sequence is often t ime consuming as it loadsand initializes mod ules and drivers required byprograms that use the system. The amount oftime sp ent executing the boot-up s equence di-rectly impacts the availability of the system; sys-tem availability is t h eMeanT imeT oF ailureMeanT imeT oF ailure + MeanT imeT oRecovery(1)Mean time to recovery is essen tially the amountof time spen d executing the boot up sequenceand it follow from eq(1), that optimizating theboot up sequence will increase the system avail-ability.In th is paper, we pose the following question:Is it possible to identify a small portion of the boot-upsequence which if optimized will result in significantimprovements in the availability of the system? Toanswer this question, we design and implementa framework to profile the boot-up sequence.This framework utilizes pc sampling to identifythe distribution of time spent in each segment ofcode executed during boot-up. The framework1identifies and r ank s code segments based on thefrequency with which we sample them.With the framework in place, we develop andapply a ballooning technique, to increase theamount of inst ruction executed in a segment.The goal of ballooning is to quantify/validatethe achievable gains from t h e o ptimization ofthe identified regions. We show that by bal-looning and increasing the number of executedinstructions we increase the boot-up time andthe rank of the modified segment. We claimthat the inverse of this holds; deflating a func-tion or decreasing the number of executed in-structions should reduce boot-up time and t h erank of the modified segments. Upon exami-nation and analysis of the top 5 segments iden-tified by the framework, we developed a fewsuggestions fo r reducing the cycles spent exe-cuting boot-up seq uence. Finally, we validateour the framework by analyzing the profiles ofcertain code se gments over different boot-up se-quences.The rest of this paper is as follows: section 2presents a brief literature survey of th e domainspace. In section 3, we discuss the implemen-tation of our framework. We then present andevaluate the results of running our frameworkon a real linux ope r ating sy stem in 4. Finally weconclude with a summary of our achievementsin section 5.2 Related WorksOur work builds o n [7] which identifies failuremodels employe d in various portions of the filesystem code. Our work looks into increasingthe availability of sy stems that employ systemrestart recovery mod el. We believe that sys-tem restart recovery is the most widely appliedfailure recovery mode l in the systems commu-nity. The systems community has worked formany years on profiling te chnique s to identifycode which requires optimization. Most of t h ework in the profiling space focuses on user-levelapplications [2], and kernel code [1]. Recently,however, [6] presented a seminal approach t oprofiling virtual machines.Our current approach differs from [2], in thatour framework doesnt discriminate betwe enuser level processes and treats all user level pro-cess as one. Unlike our architecture, the ar-chitecture presented in [2] targets specific pro-cesses and only analyse s time spent in userspace.Unlike, prior kernel profilers [1] which re-quire the kernel to load certain mod ules beforeprofiling can be initiated our approach can be-gin profile from startup time. In comparison,our me thod suffers a major draw back becausewe only profile ker n els that have been modifiedto run on the xen virtual machine. These mod-ifications alter the shape of our results and addnoise that would ot h erwise not be present in theenvironment profiled by other kernel profilers.Contrary to the approach taken in [1], we per-form software level monitoring.Finally, seminal work in [6] presents a frame-work similar to ours, however, we defer in twothings; where profiling analysis is performedand the amount of data-structures used. [6]runs profiling anlysis tools from within the pro-filed operating system w h ile we run our toolsfrom the main operating system running in do-main 0.3 ArchitectureOur profiler implementation requires changesin three different levels: the Xen Hy pervisor, theLinux ke r n el, and user-level tools. This imple-mentation supports flexibility and ease of usewith regard to acquiring profiling data.3.1 Program Counter SamplingA useful way to determine a performance pro-file is with program counter sampling. Theprogram counter contains the address of