WebQuilt: A Framework for Capturing and Visualizing theWeb ExperienceJason I. Hong and James A. LandayGroup for User Interface Research, Computer Science DivisionUniversity of California at BerkeleyBerkeley, CA 94720-1776 USA+1 510 643 7354{jasonh, landay}@cs.berkeley.eduABSTRACTWebQuilt is a web logging and visualization system that helpsweb design teams run usability tests (both local and remote) andanalyze the collected data. Logging is done through a proxy,overcoming many of the problems with server-side and client-sidelogging. Captured usage traces can be aggregated and visualizedin a zooming interface that shows the web pages people viewed.The visualization also shows the most common paths takenthrough the website for a given task, as well as the optimal pathfor that task as designated by the designer. This paper discussesthe architecture of WebQuilt and also describes how it can beextended for new kinds of analyses and visualizations.Categories and Subject DescriptorsH.1.2 [Models and Principles]: User/Machine Systems – Humanfactors; H.3.5 [Information Storage and Retrieval] OnlineInformation Services – Web-based services; H.5.2 [InformationInterfaces and Presentation] User Interfaces – Evaluation /methodology; H.5.4 [Information Interfaces and Presentation]Hypertext/Hypermedia – User issuesGeneral TermsMeasurement, Design, Experimentation, Human FactorsKeywordsusability evaluation, log file analysis, web visualization, webproxy, WebQuilt1. INTRODUCTIONThere are two problems all web designers face: understandingwhat tasks people are trying to accomplish on a website andfiguring out what difficulties people encounter in completingthese tasks. Just knowing one or the other is insufficient. Forexample, a web designer could know that someone wants to findand purchase gifts, but this isn’t useful unless the web designeralso knows what problems are preventing the individual fromcompleting the task. Likewise, the web designer could know thatthis person left the site at the checkout process, but this isn’tmeaningful unless the designer also knows that he truly intendedto buy something and is not simply browsing.There are a variety of methods for discovering what people wantto do on a website, such as structured interviews, ethnographicobservations, and questionnaires (for example, see [3]). Instead,we focus here on techniques for tackling the other problem, that isunderstanding what obstacles people are facing on a website.1Traditionally, this kind of information is gathered by runningusability tests on a website. A usability specialist brings in severalparticipants to a usability lab and asks them to complete a fewpredefined tasks. The usability engineer observes what stumblingblocks people come across and follows up with a survey and aninterview to gain more insights into the issues.The drawback to this traditional approach is that it is very timeconsuming to run usability tests with large numbers of people: ittakes a considerable amount of work to schedule participants,observe them, and analyze the results. Consequently, the datatends to reflect only a few people and is mostly qualitative. Thesesmall numbers also make it hard to cover all of the possible taskson a site. Furthermore, small samples are less convincing whenasking management to make potentially expensive changes to asite. Lastly, a small set of participants may not find the majority ofusability problems. Despite previous claims that around fiveparticipants are enough to find the majority of usability problems[14, 19], a recent study by Spool and Schroeder suggests that thisnumber may be nowhere near enough [17]. Better techniques andtools are needed to increase the number of participants and tasksthat can be managed for a usability test.Figure 1. Server-side logging is done on the web server, butthe data is available only to the owners of the server.In contrast to traditional usability testing, server log analysis (SeeFigure 1) is one way of quantitatively understanding what largenumbers of people are doing on a website. Nearly every webserver logs page requests, making server log analysis quitepopular. In fact, there are over 90 research, commercial, andfreeware tools currently available [1]. Server logging also has theadvantage of letting test participants work remotely in their ownenvironments: instead of coming to a single place, usability testWebQuilt source code and documentation can be downloaded athttp://guir.berkeley.edu/projects/webquiltPermission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and thatcopies bear this notice and the full citation on the first page. To copyotherwise, or republish, to post on servers or to redistribute to lists,requires prior specific permission and/or a fee.WWW10, May 1-5, 2001, Hong Kong.Copyright 2001 ACM 1-58113-348-0/01/0005…$5.00.participants can evaluate a website from any location on their owntime, using their own equipment and network connection.However, from the perspective of the web design team, there aretwo problems with server logs. The first is with deployment.Access to server logs are often restricted to just the owners of theweb server. This can make it difficult to analyze subsites that existon a server. For example, a company may own a single web serverwith different subsites owned by separate divisions. For the samereason, it is also impractical to do a log file analysis of acompetitor’s website. A competitive analysis is important inunderstanding what features people consider important, as well aslearning what parts of your site are easy-to-use and which are notin comparison to a competitor’s site.Figure 2. Client-side logging is done on the client computer,but requires special software running in the background orhaving a special web browser.Client-side logging has been developed to overcome thesedeployment problems. In this approach, participants remotely testa website by downloading special software that records web usage(See Figure 2). However, client-side logging has two weaknesses.First, the design team must deploy the special software and haveend-users install it. Second, this technique makes it hard toachieve compatibility with a range of operating systems and webbrowsers. What is needed is a logging technique that is easy todeploy for any website and is compatible with a number