IntroductionAI in Video GamesNeuroevolutionBuilding Machine Learning GamesConclusionCreating Intelligent Agents in GamesRisto MiikkulainenThe University of Texas at AustinAbstractGame playing has long been a central topic in artificial intelligence. Whereas early research focused on utilizing searchand logic in board games, machine learning in video games is driving much of the recent research. In video games,intelligent behavior can be naturally captured through interaction with the environment, and biologically inspiredtechniques such as evolutionary computation, neural networks, and reinforcement learning are well suited for thistask. In particular, neuroevolution, i.e. constructing artificial neural network agents through simulated evolution, hasshown much promise in many game domains. Based on sparse feedback, complex behaviors can be discovered forsingle agents and for teams of agents, even in real time. Such techniques may allow building entirely new genres ofvideo games that are more engaging and entertaining than current games, and can even serve as training environmentsfor people. Techniques developed in such games may also be widely applicable to other fields, such as robotics,resource optimization, and intelligent assistants.1 IntroductionGames have long been a popular area of artificial intelligence (AI) research, and for a good reason. They are chal-lenging yet easy to formalize, making it possible to develop new AI methods, measure how well they are working,and demonstrate that machines are capable of impressive behavior generally thought to require intelligence withoutputting human lives or property at risk.Most of the research so far has focused on games that can be described in a compact form using symbolic represen-tations, such as board and card games. The so-called “good old-fashioned artificial intelligence” (GOFAI; Haugeland1985) techniques work well with them, and to a large extent, such techniques were developed for such games. Theyhave led to remarkable successes, such as the checkers program Chinook (Schaeffer 1997) that became the worldchampion in 1994, and the chess program Deep Blue (Campbell et al. 2002) that defeated the world champion in1997, gaining significant attention to AI.Since the 1990s, the field of gaming has changed tremendously. Inexpensive yet powerful computer hardware hasmade it possible to simulate complex physical environments, resulting in an explosion of the video game industry.From modest beginnings in the 1960s (Baer 2005), the entertainment software sales have expanded to $25.4 billionworldwide in 2004 (Crandall and Sidak 2006). Video games have become a facet of many people’s lives and themarket continues to expand.Curiously, this expansion has involved little AI research. Many video games utilize no AI techniques, and thosethat do are usually based on relatively standard, labor-intensive scripting and authoring methods. The reason is thatvideo games are very different from the symbolic games. There are often many agents involved, embedded in asimulated physical environment where they interact through sensors and effectors that take on numerical rather thansymbolic values. To be effective, the agents have to integrate noisy input from many sensors, and they have to reactquickly and change their behavior during the game. The techniques that have been developed for and with symbolicgames are not well suited for video games.In contrast, machine learning (ML) techniques such as neural networks, evolutionary computing, and reinforce-ment learning are very well suited for video games. They excel in exactly the kinds of fast, noisy, numerical, statistical,and changing domains that today’s video games provide. Therefore, video games constitute an opportunity similar tothat of the symbolic games for GOFAI in 1980s and 1990s: an opportunity to develop and test ML techniques, and anopportunity to transfer the technology to industry.To appear in Proceedings of the National Academy of Engineering 20006 Conference on Frontiers of Engineering2 AI in Video GamesOne of the main challenges for AI is to create intelligent agents that adapt, i.e. change their behavior based on interac-tions with the environment, becoming more proficient in their tasks over time, and adapting to new situations as theyoccur. Such ability is crucial for deploying robots in human environments, as well as for various software agents thatlive in the Internet or serve as human assistants or collaborators.While general such systems are still beyond current technology, they are already possible in special cases. Inparticular, video games provide complex artificial environments that can be controlled, and carry perhaps the least riskto human life of any real-world application (Laird and van Lent 2000). On the other hand, such games are an importantpart of human activity, with millions of people spending countless hours on them. Machine learning can make videogames more interesting and decrease their production costs (Fogel et al. 2004). In the long run, such technology mightalso make it possible to train humans realistically in simulated adaptive environments. Video gaming is therefore animportant application of AI on its own right, and an excellent platform for research in intelligent adaptive agents.Current video games include a variety of high-realism simulations of human-level control tasks, such as navigation,combat, team and individual tactics and strategy. Some of these simulations involve traditional AI techniques such asscripts, rules, and planning (Agre and Chapman 1987; Maudlin et al. 1984). A large fraction of AI development isdevoted to path-finding algorithms such as A*-search and simple behaviors built using finite state machines. The AIis used to control the behavior of the non-player-characters (NPCs), i.e. the autonomous computer-controlled agentsin the game. Although such agents can exhibit impressive behaviors, they are often repetitive and inflexible. Indeed, alarge part of the gameplay in many games is figuring out what the AI is programmed to do, and learning to defeat it.More recently, machine-learning techniques have begun to appear in video games. This trend follows a longhistory of learning in board games, originating from Samuel’s (1959) checkers program that was based on a methodsimilar to temporal difference learning (Sutton 1988), followed by various learning methods applied to tic-tac-toe,backgammon, go, othello, and checkers
View Full Document
Unlocking...