View Full Document

14 views

Unformatted text preview:

A Technique for Enabling and Supporting Debugging of Field Failures James Clause and Alessandro Orso College of Computing Georgia Institute of Technology clause orso cc gatech edu Abstract It is difficult to fully assess the quality of software inhouse outside the actual time and context in which it will execute after deployment As a result it is common for software to manifest field failures failures that occur on user machines due to untested behavior Field failures are typically difficult to recreate and investigate on developer platforms and existing techniques based on crash reporting provide only limited support for this task In this paper we present a technique for recording reproducing and minimizing failing executions that enables and supports inhouse debugging of field failures We also present a tool that implements our technique and an empirical study that evaluates the technique on a widely used e mail client 1 Introduction Quality assurance activities such as software testing and analysis are notoriously difficult expensive and timeconsuming As a result software products are often released with faults or missing functionality In fact realworld examples of field failures experienced by users because of untested behaviors e g due to unforeseen usages are countless When field failures occur it is important for developers to be able to recreate and investigate them in house This pressing need is demonstrated by the emergence of several crash reporting systems such as Microsoft s error reporting systems 13 and Apple s Crash Reporter 1 Although these techniques represent a first important step in addressing the limitations of purely inhouse approaches to quality assurance they work on limited data typically a snapshot of the execution state and can at best identify correlations between a crash report and data on other known failures In this paper we present a novel technique for reproducing and investigating field failures that addresses the limitations of existing approaches Our technique works in three phases intuitively illustrated by the scenario in Figure 1 In the recording phase while users run the software the tech nique intercepts and logs the interactions between application and environment and records portions of the environment that are relevant to these interactions If the execution terminates with a failure the produced execution recording is stored for later investigation In the minimization phase using free cycles on the user machines the technique replays the recorded failing executions with the goal of automatically eliminating parts of the executions that are not relevant to the failure In the replay and debugging phase developers can use the technique to replay the minimized failing executions and investigate the cause of the failures e g within a debugger Being able to replay and debug real field failures can give developers unprecedented insight into the behavior of their software after deployment and opportunities to improve the quality of their software in ways that were not possible before To evaluate our technique we implemented it in a prototype tool called ADDA Automated Debugging of Deployed Applications and used the tool to perform an empirical study The study was performed on PINE 19 a widelyused e mail client and involved the investigation of failures caused by two real faults in PINE The results of the study are promising Our technique was able to 1 record all executions of PINE and two other subjects with a low time and space overhead 2 completely replay all recorded executions and 3 perform automated minimization of failing executions and obtain shorter executions that manifested the same failures as the original executions Moreover we were able to replay the minimized executions within a debugger which shows that they could have actually been used to investigate the failures The contributions of this paper are A novel technique for recording and later replaying executions of deployed programs An approach for minimizing failing executions and generating shorter executions that fail for the same reasons A prototype tool that implements our technique An empirical study that shows the feasibility and effectiveness of the approach In House In the Field Site 1 e or ing p ase inimi ation p ase 0on1line3 0off1line3 rele Softwa ase r s up e date s Software Software Software Minimized recording ADDA Recording Tool eplay an e ugging p ase 0in1 ouse3 Minimized environment Internet Minimized log sers ADDA Replay tool Software Software e9elopers Execution recording Execution recording Environment data Environment data Local repository ex Fiel ec d ut m ion ini re miz co ed rd ing s Event log Minimized recording ADDA Minization Tool Event log Remote repository Site 2 Site n Figure 1 An intuitive scenario of usage of our technique 2 Related Work This work encompasses several areas We present the most related efforts organized into categories Record and replay The techniques most closely related to our approach are those that record and replay executions for testing or debugging Some of these techniques perform deterministic replay debugging that is replay of executions that led to a crash e g 3 9 14 15 21 Various commercial and research tools record and replay user interactions with a software product for regression testing e g 11 22 23 Unlike our approach most of these techniques are designed to be used during in house testing or debugging The overhead they impose on space time or infrastructure required for recording is reasonable for their intended use but would make them impractical for use on deployed software The few record and replay techniques that are either defined to operate in the field e g BugNet 14 or may be efficient enough to be used in the field e g 9 21 require a specialized operating system or hardware support which considerably limits their applicability in the short term More recently several researchers presented techniques for recording executions of Java subsystems 5 16 17 20 These approaches are heavily based on Java s characteristics and target the recording of subsystems only It would be difficult to adapt them to work in a more general context Moreover most of these approaches are defined to be used in house for regression testing and impose an overhead that would prevent their use on deployed software Remote data collection Pavlopoulou and Young propose residual testing 18 which continuously monitors test obligations not


Access the best Study Guides, Lecture Notes and Practice Exams

Loading Unlocking...
Login

Join to view LECTURE NOTES and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view LECTURE NOTES and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?