Analyzing Multi-agent Activity LogsUsing Process Mining TechniquesA. Rozinat1,2, S. Zickler2, M. Veloso2, W.M.P. van der Aalst1, and C.McMillenAbstractDistributed autonomous robotic systems exhibit complex behavior that—although programmed, but due to the impact of the environment—only ma-terializes as the process unfolds. Thus, the actual behavior of such a systemcannot be known in advance but must be observed to be evaluated or veri-fied. In this paper we propose to use process mining techniques to extract,compare, and enhance models of the actual behavior of a multi-agent roboticsystem through analyzing collected log data. We use the example of robotsoccer as such a multi-agent robotic system, and we demonstrate which typesof analysis are currently possible in the context of the process mining toolset ProM.1 IntroductionRobotic systems are growing more and more complex as they seek, for exam-ple, to be self-reconfiguring, self-organizing, or working in teams. While theirbehavioral logic is of course programmed, and, thus, in principle predictable,the more autonomous a robot grows, or the more it adapts to its environment,the more it is true that the actual behavior of the system cannot really beknown in advance anymore. For example, in a team of robots, the overall sys-tem behavior is determined through interaction among multiple robots. Or,if robots interact with humans, their actions are influenced by the actions ofthe human. Thus, the question whether this overall behavior corresponds tothe intended system behavior can only be answered through observation.Process mining techniques use log data to analyze observed processes andhave been successfully applied to real-life logs from, e.g., hospitals, banks,municipalities etc. (see [2] for one of many real-life applications). The basicidea of process mining is to discover, monitor and improve real processes (i.e.,Information Systems Group, Eindhoven University of Technology, NL-5600 MB,Eindhoven, The Netherlands. {a.rozinat,w.m.p.v.d.aalst}@tue.nl · Computer Sci-ence Department, Carnegie Mellon University, Pittsburgh PA 15213-3890, USA.{szickler,veloso,mcmillen}@cs.cmu.edu1not assumed processes) by extracting knowledge from event logs. Today manyof the activities occurring in processes are either supported or monitored byinformation systems. However, process mining is not limited to informationsystems and can also be used to monitor other operational processes or sys-tems, such as complex X-ray machines, high-end copiers, or web services.The common denominator in the various applications of process mining isthat there is a notion of a process and that the occurrences of activities arerecorded in so-called event logs [1].In this paper, we use log data collected by the CMDragons team duringthe international robot soccer competition ’RoboCup’ 2007 to investigate theapplicability of process mining techniques to a multi-agent robotic system. Inthe context of robot soccer, the motivation for analyzing log data is two-fold:• Self-analysis: While detailed logs are recorded during the competitions,the evaluation of a game is carried out mostly through watching theaccompanying video recordings. A wealth of detailed data are available,but usually they are not analyzed in a structured way. We argue thata systematic and more high-level analysis of these log data can help toobtain an overall picture, and to, for example, discover transient faultystates that—due to being held only for a short time—are difficult to beobserved by the human eye.• Opponent analysis: Obtaining behavioral models of the opponentrobots is even more interesting as their intentions are hidden and theonly way to derive knowledge about their strategic behavior is throughobservation. We envision the application of process mining techniques incombination with activity recognition [9, 7], which is able to identify high-level actions (for example that a robot is “attacking”) based on low-levelbehaviors (such as the robot’s position and velocity).Without loss of generality, we can restrict ourselves to self-analysis, sincethe described techniques can be equally applied for the purpose of opponentanalysis given that appropriate activity recognition mechanisms are in place.The paper is organized as follows. First, we describe the domain of robotsoccer that is used in the remainder of this paper as an example of a multi-robot system (Section 2). Then, we introduce process mining and explainhow process mining can be applied in the context of this example domain(Section 3). Next, the log data that are used as input for the process miningtechniques are described (Section 4) and some illustrative analysis results arepresented (Section 5). Section 6 concludes the paper.2 Robot Soccer: A Multi-agent SystemBehavioral multi-robot systems are control architectures where multipleagents coordinate the execution of different individual tactical approaches,2called behaviors, typically in order to achieve a common goal. One particu-larly interesting behavioral multi-agent domain, which we are going to useas the data source for our experiments, is the RoboCup Small Size League.Here, the objective is to have two teams of small robots compete in a game ofminiature soccer, without human intervention. In the Small-Size league, eachteam normally consists of five homogeneous, omni-directional robots whichare remotely controlled by an off-board computer. The computer obtainsits observations from two overhead cameras mounted above the soccer fieldwhich are then processed to provide very accurate estimates of positions andorientations of all the players on the field. Challenges of this domain includeits very fast pace, and its complex tactical properties. A scene from a typicalrobocup game is shown in Figure 1.Fig. 1 A scene of a RoboCup Small-Size League game.This paper utilizes real game data collected by Carnegie Mellon Univer-sity’s Small-Size team “CMDragons” [5]. The software architecture of theteam’s offboard control system is shown in Figure 2. The server componentin this diagram performs computer vision and manages communication withthe robots. The intelligence of the system arises from the soccer componentwhich embodies all autonomous decision making processes. The heart of thesoccer component is its behavioral control architecture, known as “Skills,Tactics, and Plays” (STP) [4]. Within STP, multi-agent coordination andteam-based strategic decisions are