The Assistment Builder: A Rapid Development Tool for ITSTerrence E. TURNER, Michael A. MACASEK, Goss NUZZO-JONES, Neil T.HEFFERNANWorcester Polytechnic Institute100 Institute Rd, Worcester, MA 01609(508) [email protected], [email protected], [email protected], [email protected] KOEDINGERHuman-Computer Interaction InstituteCarnegie Mellon University, Pittsburgh, PA, USAAbstract. Intelligent Tutoring Systems are notoriously costly to construct [1], and requirePhD level experience in cognitive science and rule based programming. The goal of thisresearch was to ease the development process for building pseudo-tutors [5], which are ITSconstructs that mimic cognitive tutors but are limited in that they only work for a singleproblem. The Assistment Builder is a system designed to rapidly develop, test, and deploysimple pseudo-tutors. These tutors provide a simple cognitive model based upon a stategraph tailored to a specific problem. These tutors offer many of the features of rule-basedtutors, but without the expensive creation time. The system simplifies the process of tutorconstruction to allow users with little or no ITS experience to develop content. The systemprovides a web-based interface as a means to build and store these simple tutors we havecalled Assistments. This paper describes our attempt to make the process of developingcontent easy for teachers. We present some evidence to suggest that these novice users candevelop a tutor for a problem in under thirty minutes.1.0 IntroductionThis research aims to develop tools for the rapid development and deployment of IntelligentTutoring Systems (ITS). Specifically, this research focused on so-called “pseudo-tutors”that are a simplification of cognitive rule-based tutors [5]. Model tracing rule-based tutors[1] have been shown to be effective [6], but development time on them is highlyprohibitive, from 100-1000 hours of development time per hour of content [7][1].Development also requires a very specialized knowledge set. Tutor developers are requiredto be expert system programmers, in addition to developing the cognitive model, to saynothing of being a content expert. Another aim of this research was to make our toolsaccessible to novices, with no programming experience, and less than an hour of training.A pseudo-tutor is a simplified cognitive model based on a state graph. Studentactions trigger transitions in the graph, and the current state of the problem is stored by thegraph. Pseudo-tutors have nearly identical behavior to a rule-based tutor, but suffer fromhaving no ability to generalize to different problems [4]. This pseudo-tutor approach allowsfor predicted behaviors and provides feedback based on those behaviors. We also combinedthis state graph with a conceptually broader branching structure referred to as scaffolding.Scaffolding provides sub-problems to the initial question, often designed to address specificconcepts within the initial question. This allows for a higher-level of predicted actions to behandled.1.1 Purpose of the Assistment BuilderThe Assistment Builder is an application supporting the Assistment Project [8]. We soughtto create a tool that would provide a simple web-based interface for creating these pseudo-tutors that could rapidly be deployed across the web, and if errors were found with the tutor,bug-fixing or correction would be quick and simple. The tool had to be usable by someonewith no programming experience or ITS background. We wanted the teachers in the publicschool system to be able to build pseudo-tutors. These pseudo-tutors are often referred to asAssistments, but the term is not limited to pseudo-tutors.A secondary purpose of the Assistment Builder was to aid the construction of aTransfer Model. A Transfer Model is a cognitive model construct divorced from specifictutors. The Transfer Model is a directed graph of knowledge components representingspecific concepts that a student could learn. This allows us to maintain a complex cognitivemodel of the student without necessarily involving a production rule system.When a user first begins to use the Assistment Builder they will be greeted by thestandard blank skeleton question. The user can enter the question text, images, answers, andhint messages to complete the root question. After these steps the appropriate scaffolding isadded. The question layout is separated into several views the Main View, All Answer View,Correct Answer View, Incorrect Answer View, Hints View, and Transfer Model View.Together these views allow a user to highly customize their question and its subsequentscaffolding.2.0 MethodsTo analyze the effectiveness of the Assistment Builder, we developed a system to log theactions of an author. Each action is recorded with associated meta-data, including author,timestamps, the specific series of problems being worked on, and data specific to eachaction. The authors were asked to build original items and keep track of roughly how muchtime spent on each item for corroboration. The authors were also asked to create “morphs,”a term used to indicate a new problem that had a very similar setup to an existing problem.“Morphs” are usually constructed by loading the existing problem into the AssistmentBuilder, altering it, and saving it with a different name. This allows rapid contentdevelopment for testing transfer between problems. We wanted to compare thedevelopment time for original items to that of “morphs” [8].Another trial of the Assistment Builder with less rigorous methodology was testinghow authors with little experience would react to the software. To test the usability of theAssistment Builder, we were able to provide the software to two high-school teachers in theWorcester, Massachusetts area. These teachers were computer literate, but had no previousexperience with intelligent tutoring systems, or creating mathematics educational software.Our tutorial consisted of demonstrating the creation of a problem using the AssistmentBuilder, then allowing the teacher to create their own with an experienced observer toanswer questions.3.0 Results & AnalysisPrior to the implementation of logging within the Assistment Builder, we obtainedencouraging anecdotal results of the software’s use. A high-school mathematics teacher wasable to create 15 items and morph each one, resulting in 30 Assistments over severalmonths. Her training consisted of approximately four hours spread over two days in