Bootstrapping Novice Data:Semi-Automated Tutor Authoring Using Student Log FilesBruce M. McLaren, Kenneth R.Koedinger, Mike SchneiderHuman-Computer Interaction InstituteCarnegie Mellon UniversityPittsburgh, PA [email protected], [email protected]@cs.cmu.eduAndreas Harrer, Lars BollenCollide Research GroupUniversity Duisburg-EssenDuisburg, [email protected], [email protected] potentially powerful way to aid in the authoring of intelligent tutoring systems is to directlyleverage student interaction log data. While problem-solving data has been used in the past toguide the development of tutors, such data has not typically been used as a means to directlyconstruct an initial tutoring system model. We propose an approach called bootstrapping novicedata (BND) in which a problem-solving tool is integrated with tutor development softwarethrough log files and that integration is then used to create the beginnings of a tutor for the tool.We describe an initial implementation of the BND approach in which Cool Modes, a collaborativesoftware tool, is integrated with the Behavior Recorder, tutor-authoring software that supportsdevelopment by demonstration. A key to this implementation is a component-based approach inwhich complementary pieces of software are integrated with little or no change to either softwarecomponent. We argue that more tutors could be built, and with substantial time savings, using thisapproach. We discuss some of the lessons learned from this initial effort and from applying thecomponent-based approach, as well as some data analyses that could eventually be performedusing the data collected during BND.Keywords: Intelligent Tutors, Authoring Tools, Log files, Collaborative LearningIntroductionIntelligent Tutoring Systems (ITS) have often been developed by programmers having expertise in cognitivemodelling and/or Artificial Intelligence together with domain experts having experience and a depth ofknowledge with a particular task. This approach is subject to problems, such as the so called "expert blind spot"in which those who are accomplished in a domain fail to recognize which aspects of that domain might provedifficult to novices (Nathan et al., 2001). An expert’s input is certainly important to the development of anintelligent tutor, but there is also much to be gained by capturing and analyzing the behavior of novices (cf,Lovett, 1998).As an alternative to the traditional approach to tutor development, we propose an approach that leveragesactual problem solving data not only to guide tutor design, as has been done before (cf, Koedinger and Terao,2002), but also to contribute directly to tutor implementation. While others have used student data to, forinstance, tune knowledge tracing parameters (Corbett et al., 2000), we intend to use student (as well as expert)data as the fundamental basis for model development. In our approach, called bootstrapping novice data (BND),we provide students with a computer-based tool, let them attempt to solve problems with the tool, and record thatproblem-solving activity in a tutor-specific representation. This integrated record of student activity helps in twoways: (1) we learn a great deal about students’ problem-solving approaches, both good and bad, and (2) itprovides the initial representational structure for tutor implementation. In this research we are particularlyinterested in how we can develop tutors by leveraging the successes, errors, and inefficiencies of a group oflearners. Our initial step has been to develop a prototype integration between a system for authoringcollaborative modeling software, Cool Modes (COllaborative Open Learning and MODEling System) (Pinkwart,2003), and a tutor authoring environment, the Cognitive Tutor Authoring Tools (CTAT) (Koedinger et al.,2004). The Cool Modes software generates computer log files of student activities that are, in turn, used byCTAT as an initial representation of the tutor. In this paper, we discuss how we have effected this integration andhow we could use this model as a way of collecting student data to create an initial, skeletal tutoring system.An important underpinning of this work is the notion of component-based development. Our approach takesan existing software application and integrates it, with little or no modification, with a tutor or tutor agent. Usingoff-the-shelf or pre-existing software as the basis for building tutoring systems could result in substantial timesavings and thus more tutor development, as compared to the traditional approach of building tutors "fromscratch" (Ritter and Koedinger, 1996; McArthur et al., 1996). With the component-based approach, tutordevelopers are able to leverage complementary pieces of software and focus attention on pedagogical techniquesand technology of the tutor piece independent of domain-specific user interfaces, representations, andfunctionality of the tool piece.The Pittsburgh Advanced Cognitive Tutor (PACT) lab at CMU has been working since the mid-1990s tofully realize the component-based integration model. The project has proven more challenging than originallyanticipated. More generally, the educational object economy (http://www.eoe.org/) and other similar effortsshowed great promise initially, but have not caught on. Part of the challenge is to get the component-basedapproach right. In particular, defining open, general, and well-structured software interfaces is a non-trivial task,and systems not originally designed as "components" are hard to integrate. Even if systems are designed ascomponents, it is not always possible to anticipate in advance the input and output requirements of acomplementary component. Another challenge is accumulating a critical mass of component producers andconsumers so that the approach can pay off. Nevertheless, we believe the component-based approach willultimately be the primary means of deploying computer-based tutoring, and thus we continue to push this as acritical agenda item in the learning technology community.Bootstrapping Novice Data: Creating the Initial Representation of a TutorThe process that we describe here,