Virtual Babel: Towards Context-Aware Machine Translation in VirtualWorldsYing Zhang and Nguyen BachCarnegie Mellon University23 S. Akron Rd.Moffett Field, CA{joy+,nbach+}@cs.cmu.eduAbstractIn this paper, we describe our ongoing re-search project of Virtual Babel, a context-aware machine translation system for SecondLife, one of the most popular virtual worlds.We augment the Second Life viewer to in-tercept the incoming/outgoing chat messagesand reroute the message to a statistical ma-chine translation server. The returned trans-lations are appended to the original text mes-sage to help users to understand the foreignlanguage. Virtual Babel provides a platformto study cross-lingual conversations facilitatedby machine translation in virtual worlds andwe observe interesting phenomena that are notpresent in document translations. Virtual Ba-bel is aware of the non-verbal context of theconversation. Language model and translationmodels are trained from collected conversa-tions and are used to generate translations ac-cording to observed non-verbal context of theconversation.1 IntroductionVirtual world is fast becoming a favorite venue foronline learning, collaborating, and networking. Justas the real world, users in the virtual world comefrom different background and speak different lan-guages. Even if some users know more than onelanguages, it is still more natural for them to usetheir native languages to communicate. Thus, thelanguage barrier hinders the communication in thevirtual world just as it does in the real world.In recent years, machine translation (MT) tech-nologies have been greatly improved. In particular,phrase-based statistical machine translation systems(Och et al., 1999; Koehn et al., 2003) have advancedto a state that the translation quality for certain do-mains (e.g. broadcasting news) and certain languagepairs (e.g. Spanish-English) is acceptable to users.The publicly available translation services such asthe Google Translation API make it possible to de-velop translation plug-ins for various applicationsincluding Skype, MSN and Google Talk.These translation services create great interestsin the community of using machine translation tobridge the language gaps in cyber communication.These general purpose machine translation servicesare usually trained from bilingual text like broad-casting news and parliamentary debate data whichis of different genre compared to the online chat.Such mismatch in genre usually results in sub op-timal translation quality. Another problem of usinggeneric translation services is that these services arecontext independent. In other words, a sentence hasthe same translation no matter where it occurs. Asshown in the following sections, ambiguities in nat-ural language can only be resolved with the context.In this paper, we describe the Virtual Babelproject, an ongoing research effort of developing acontext-aware machine translation system for vir-tual worlds. On one hand, machine translation cangreatly help the multilingual communication insidethe virtual world due to the non-critical nature of on-line chats. On the other hand, virtual worlds allowMT systems to explore the non-verbal context of theconversation in a much easier way than in real world.This makes it feasible for us to study how contextinfluences the language usage and how MT systemcould make use of context information to improvethe translation quality.2 Virtual WorldsVirtual worlds, such as Second Life (SL), World ofWarcraft and Kaneva are computer-based environ-ments where real-life users inhabit and interact viaavatars. Virtual worlds are close-to-real simulationof the real world and users can experience telepres-ence to a certain degree. Users can use both instanttext message (IM) and real-time voice chat to com-municate with other users in the 3D environment.Users can participate many virtual activities in vir-tual worlds including sight-seeing, talk to other peo-ple, dancing, listen to live music and attend live con-cert, attend lectures in open university, building andcreating things, doing business, shopping and role-playing games.We are particularly interested in providing trans-lation services for educational activities in virtualworlds. Virtual worlds provide an alternative, moreengaging platform for education in cyber-space. A3D virtual world provides students with a supportivecommunity. Feeling part of a community of learnershas a direct impact, not only on retention, but alsoon students’ perception of successful university ex-periences (Wellman and Kahne, 1993; Wehlage etal., 1998).Many educators have discovered the unique pos-sibilities offered by utilizing these virtual worlds todevelop new forms of education. A report funded bythe Eduserv Foundation estimates that some three-quarters of UK universities are actively develop-ing or using SL, at the institutional, departmentaland/or individual academic level1. Harvard Uni-versity, Texas State University, and Stanford Uni-versity have set up virtual campuses where studentscan meet, attend classes, and create content together.Using virtual world applications, students aroundthe globe can “sit” in the same classroom without theneed to build physical campuses thousands of milesaway.According to the Key Metrics report published bySecond life on June 11, 20072, there are 507,8441http://www.eduserv.org.uk/foundation/studies/slsnapshots2http://blog.secondlife.com/2007/06/12/may-2007-key-metrics-publishedactive users coming from 100 countries and regions.On average each active user spends 36 hours in thevirtual world. Table 1 shows the 20 countries withthe most active users in Second Life. Though thestatistics shown here include all activities in SL, weassume that the educational activities have a similardistribution among users’ from different parts of theworld. The need of a universal translation system isobvious given the fact that not all users speak En-glish and even they do, a user would prefer to usehis/her native language.3 Virtual Babel Translation ServiceJust as in real life, users are most comfortable speak-ing their native languages. The demand for auto-matic translation in virtual worlds is as strong as inthe real world. Several text translation tools havebeen developed based on existing online translationservices such as Babel Fish and Google LanguageTranslator. Users type in messages in their own lan-guages and the machine translation results in the for-eign language are overlayed in the chat