Measuring and Characterizing End-to-EndInternet Service PerformanceLUDMILA CHERKASOVAHewlett-Packard LaboratoriesYUN FUDuke UniversityWENTING TANGHewlett-Packard LaboratoriesandAMIN VAHDATDuke UniversityFundamental to the design of reliable, high-performance network services is an understanding ofthe performance characteristics of the service as perceived by the client population as a whole.Understanding and measuring such end-to-end service performance is a challenging task. Cur-rent techniques include periodic sampling of service characteristics from strategic locations in thenetwork and instrumenting Web pages with code that reports client-perceived latency back to aperformance server. Limitations to these approaches include potentially nonrepresentative accesspatterns in the first case and determining the location of a performance bottleneck in the second.This paper presents EtE monitor, a novel approach to measuring Web site performance. Oursystem passively collects packet traces from a server site to determine service performance char-acteristics. We introduce a two-pass heuristic and a statistical filtering mechanism to accuratelyreconstruct different client page accesses and to measure performance characteristics integratedacross all client accesses. Relative to existing approaches, EtE monitor offers the following bene-fits: i) a latency breakdown between the network and server overhead of retrieving a Web page,ii) longitudinal information for all client accesses, not just the subset probed by a third party,iii) characteristics of accesses that are aborted by clients, iv) an understanding of the performancebreakdown of accesses to dynamic, multitiered services, and v) quantification of the benefits ofnetwork and browser caches on server performance. Our initial implementation and performanceanalysis across three different commercial Web sites confirm the utility of our approach.A short version of this article was published in USENIX’2002. A. Vahdat and Y. Fu are supported inpart by research grant from HP and by the National Science Foundation (EIA-9972879). A. Vahdatis also supported by an NSF CAREER award (CCR-9984328).Author’s addresses: L. Cherkasova and W. Tang, Hewlett-Packard Laboratories, 1501 Page MillRoad, Palo Alto, CA 94303; email: {lucycherkasova,wenting tang}@hp.com; Y. Fu and A. Vahdat,Department of Computer Science, Duke University, Durham, NC 27708; email: {fu,vahdat}@cs.duke.eduPermission to make digital or hard copies of part or all of this work for personal or classroom use isgranted without fee provided that copies are not made or distributed for profit or direct commercialadvantage and that copies show this notice on the first page or initial screen of a display alongwith the full citation. Copyrights for components of this work owned by others than ACM must behonored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers,to redistribute to lists, or to use any component of this work in other works requires prior specificpermission and/or a fee. Permissions may be requested from Publications Dept., ACM, Inc., 1515Broadway, New York, NY 10036 USA, fax: +1 (212) 869-0481, or [email protected]°2003 ACM 1533-5399/03/1100-0347 $5.00ACM Transactions on Internet Technology, Vol. 3, No. 4, November 2003, Pages 347–391.348•L. Cherkasova et al.Categories and Subject Descriptors: C.2.3 [Computer-Communication Networks]: NetworkOperations—Network monitoring; C.2.4 [Computer-Communication Networks]: DistributedSystems—Client/server; C.2.5 [Computer-Communication Networks]: Local and Wide-AreaNetworks—Internet; C.4 [Performance of Systems]: Measurement techniques, Modeling tech-niques, Design studies; D.2.5 [Software Engineering]: Testing and Debugging—Monitors; D.2.8[Software Engineering]: Metrics—Performance measuresGeneral Terms: Measurement, PerformanceAdditional Key Words and Phrases: End-to-end service performance, network packet traces, passivemonitoring, QoS, reconstruction of web page composition, web site performance1. INTRODUCTIONRecent technology trends are increasingly leading to an environment whereservice, reliability, and robustness are eclipsing raw system behavior as theprimary evaluation metrics for distributed services. First, the Internet is in-creasingly being used to deliver important services in support of business, gov-ernment, education, and entertainment. At the same time, mission critical op-erations related to scientific instrumentation, military operations, and healthservices, are making increasing use of the Internet for delivering informationand distributed coordination. Second, accessing a particular logical service (e.g.,a news service or a bank account) typically requires the complex interaction ofmultiple machines and physical services (e.g., a database, an application server,a Web server, request routing, etc.) often spread across the network. Finally, thebaseline performance of servers and networks continues to improve at exponen-tial rates, often making available performance plentiful in the common case. Atthe same time, access to network services is inherently bursty, making order ofmagnitude spikes in request load relatively common.A first step in building reliable and robust network services is tracking andunderstanding the performance of complex services across a diverse and rapidlychanging client population. In a competitive landscape, such understandingis critical to continually evolving and engineering Internet services to matchchanging demand levels and client populations. By understanding current ser-vice access characteristics, sites might employ software to dynamically adaptto current network conditions, for example by reducing bandwidth overhead bytranscoding Web page content, by leveraging additional replicas at appropri-ate locations in a content distribution network, or by reducing the data qual-ity of query results to dynamic services, for instance, by sampling databasecontents.In general, a Web page is composed of an HTML file and several embeddedobjects such as images. A browser retrieves a Web page by issuing a series ofHTTP requests for all objects. However, HTTP does not provide any means todelimit the beginning or the end of a Web page. Since client-perceived Webserver responses correspond to retrieval of Web pages, effectively measuringand analyzing the Web page download process is a critical and challengingproblem in evaluating end-to-end