This article was downloaded by: [Arizona State University]On: 07 November 2011, At: 11:24Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: MortimerHouse, 37-41 Mortimer Street, London W1T 3JH, UKEducational PsychologistPublication details, including instructions for authors and subscription information:http://www.tandfonline.com/loi/hedp20The Relative Effectiveness of Human Tutoring,Intelligent Tutoring Systems, and Other TutoringSystemsKURT VanLEHN aa Computing, Informatics and Decision Systems Engineering Arizona State UniversityAvailable online: 17 Oct 2011To cite this article: KURT VanLEHN (2011): The Relative Effectiveness of Human Tutoring, Intelligent Tutoring Systems,and Other Tutoring Systems, Educational Psychologist, 46:4, 197-221To link to this article: http://dx.doi.org/10.1080/00461520.2011.611369PLEASE SCROLL DOWN FOR ARTICLEFull terms and conditions of use: http://www.tandfonline.com/page/terms-and-conditionsThis article may be used for research, teaching, and private study purposes. Any substantial or systematicreproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form toanyone is expressly forbidden.The publisher does not give any warranty express or implied or make any representation that the contentswill be complete or accurate or up to date. The accuracy of any instructions, formulae, and drug dosesshould be independently verified with primary sources. The publisher shall not be liable for any loss, actions,claims, proceedings, demand, or costs or damages whatsoever or howsoever caused arising directly orindirectly in connection with or arising out of the use of this material.Educational Psychologist, 46(4), 197–221, 2011CopyrightCDivision 15, American Psychological AssociationISSN: 0046-1520 print / 1532-6985 onlineDOI: 10.1080/00461520.2011.611369The Relative Effectiveness of HumanTutoring, Intelligent Tutoring Systems, andOther Tutoring SystemsKurt VanLehnComputing, Informatics and Decision Systems EngineeringArizona State UniversityThis article is a review of experiments comparing the effectiveness of human tutoring, com-puter tutoring, and no tutoring. “No tutoring” refers to instruction that teaches the samecontent without tutoring. The computer tutoring systems were divided by their granularityof the user interface interaction into answer-based, step-based, and substep-based tutoringsystems. Most intelligent tutoring systems have step-based or substep-based granularities ofinteraction, whereas most other tutoring systems (often called C AI, CBT, or C AL systems)have answer-based user interfaces. It is widely believed as the granularity of tutoring decreases,the effectiveness increases. In particular, when compared to No tutoring, the effect sizes ofanswer-based tutoring systems, intelligent tutoring systems, and adult human tutors are be-lievedtobed = 0.3, 1.0, and 2.0 respectively. This review did not confirm these beliefs.Instead, it found that the effect size of human tutoring was much lower: d = 0.79. Moreover,the effect size of intelligent tutoring systems was 0.76, so they are nearly as effective as humantutoring.From the earliest days of computers, researchers have strivedto develop computer tutors that are as effective as humantutors (S. G. Smith & Sherwood, 1976). This review is aprogress report. It compares computer tutors and human tu-tors for their impact on learning gains. In particular, thereview focuses on experiments that compared one type oftutoring to another while attempting to control all other vari-ables, such as the content and duration of the instruction.The next few paragraphs define the major types of tutor-ing reviewed here, starting with human tutoring. Currentbeliefs about the relative effectiveness of the types of tutor-ing are then presented, followed by eight common explana-tions for these beliefs. Building on these theoretical points,the introduction ends by formulating a precise hypothesis,which is tested with meta-analytic methods in the body of thereview.Although there is a wide variety of activities encompassedby the term “human tutoring,” this article uses “human tu-toring” to refer to an adult, subject-matter expert workingsynchronously with a single student. This excludes manyCorrespondence should be addressed to Kurt VanLehn, Computing, In-formatics and Decision Systems Engineering, Arizona State University, POBox 878809, 699 South Mill Avenue, Tempe, AZ 85287-8809. E-mail:[email protected] kinds of human tutoring, such as peer tutoring, cross-age tutoring, asynchronous online tutoring (e.g., e-mail orforums), and problem-based learning where a “tutor” workswith small group of students. Perhaps the major reason whycomputer tutor developers have adopted adult, one-on-one,face-to-face tutoring as their gold standard is a widely heldbelief that such tutoring is an extremely effective method ofinstruction (e.g., Graesser, VanLehn, Rose, Jordan, & Harter,2001). Computer developers are not alone in their belief. Forinstance, parents sometimes make great sacrifices to hire aprivate tutor for their child.Even within this restricted definition of human tutoring,one could make distinctions. For instance, synchronous hu-man tutoring includes face-to-face, audio-mediated, and text-mediated instantaneous communication. Human tutoring canbe done as a supplement to the students’ classroom instruc-tion or as a replacement (e.g., during home schooling). Tu-toring can teach new content, or it can also be purely reme-dial. Because some of these distinctions are rather difficultto make precisely, this review allows “human tutoring” tocover all these subcategories. The only proviso is that thesevariables be controlled during evaluations. For instance, ifthe human tutoring is purely remedial, then the computer tu-toring to which it is compared should be purely remedial aswell. In short, the human tutoring considered in this reviewDownloaded by [Arizona State University] at 11:24 07 November 2011198 VANLEHNincludes all kinds of one-on-one, synchronous tutoring doneby an adult, subject-matter expert.In contrast to human tutoring, which is treated as onemonolithic type, two technological types of computer tutor-ing are traditionally distinguished. The first type is charac-terized by giving students immediate feedback and hints ontheir answers. For instance, when asked to solve a