Creating Intelligent Agents in Games Risto Miikkulainen The University of Texas at Austin Abstract Game playing has long been a central topic in artificial intelligence Whereas early research focused on utilizing search and logic in board games machine learning in video games is driving much of the recent research In video games intelligent behavior can be naturally captured through interaction with the environment and biologically inspired techniques such as evolutionary computation neural networks and reinforcement learning are well suited for this task In particular neuroevolution i e constructing artificial neural network agents through simulated evolution has shown much promise in many game domains Based on sparse feedback complex behaviors can be discovered for single agents and for teams of agents even in real time Such techniques may allow building entirely new genres of video games that are more engaging and entertaining than current games and can even serve as training environments for people Techniques developed in such games may also be widely applicable to other fields such as robotics resource optimization and intelligent assistants 1 Introduction Games have long been a popular area of artificial intelligence AI research and for a good reason They are challenging yet easy to formalize making it possible to develop new AI methods measure how well they are working and demonstrate that machines are capable of impressive behavior generally thought to require intelligence without putting human lives or property at risk Most of the research so far has focused on games that can be described in a compact form using symbolic representations such as board and card games The so called good old fashioned artificial intelligence GOFAI Haugeland 1985 techniques work well with them and to a large extent such techniques were developed for such games They have led to remarkable successes such as the checkers program Chinook Schaeffer 1997 that became the world champion in 1994 and the chess program Deep Blue Campbell et al 2002 that defeated the world champion in 1997 gaining significant attention to AI Since the 1990s the field of gaming has changed tremendously Inexpensive yet powerful computer hardware has made it possible to simulate complex physical environments resulting in an explosion of the video game industry From modest beginnings in the 1960s Baer 2005 the entertainment software sales have expanded to 25 4 billion worldwide in 2004 Crandall and Sidak 2006 Video games have become a facet of many people s lives and the market continues to expand Curiously this expansion has involved little AI research Many video games utilize no AI techniques and those that do are usually based on relatively standard labor intensive scripting and authoring methods The reason is that video games are very different from the symbolic games There are often many agents involved embedded in a simulated physical environment where they interact through sensors and effectors that take on numerical rather than symbolic values To be effective the agents have to integrate noisy input from many sensors and they have to react quickly and change their behavior during the game The techniques that have been developed for and with symbolic games are not well suited for video games In contrast machine learning ML techniques such as neural networks evolutionary computing and reinforcement learning are very well suited for video games They excel in exactly the kinds of fast noisy numerical statistical and changing domains that today s video games provide Therefore video games constitute an opportunity similar to that of the symbolic games for GOFAI in 1980s and 1990s an opportunity to develop and test ML techniques and an opportunity to transfer the technology to industry To appear in Proceedings of the National Academy of Engineering 20006 Conference on Frontiers of Engineering 2 AI in Video Games One of the main challenges for AI is to create intelligent agents that adapt i e change their behavior based on interactions with the environment becoming more proficient in their tasks over time and adapting to new situations as they occur Such ability is crucial for deploying robots in human environments as well as for various software agents that live in the Internet or serve as human assistants or collaborators While general such systems are still beyond current technology they are already possible in special cases In particular video games provide complex artificial environments that can be controlled and carry perhaps the least risk to human life of any real world application Laird and van Lent 2000 On the other hand such games are an important part of human activity with millions of people spending countless hours on them Machine learning can make video games more interesting and decrease their production costs Fogel et al 2004 In the long run such technology might also make it possible to train humans realistically in simulated adaptive environments Video gaming is therefore an important application of AI on its own right and an excellent platform for research in intelligent adaptive agents Current video games include a variety of high realism simulations of human level control tasks such as navigation combat team and individual tactics and strategy Some of these simulations involve traditional AI techniques such as scripts rules and planning Agre and Chapman 1987 Maudlin et al 1984 A large fraction of AI development is devoted to path finding algorithms such as A search and simple behaviors built using finite state machines The AI is used to control the behavior of the non player characters NPCs i e the autonomous computer controlled agents in the game Although such agents can exhibit impressive behaviors they are often repetitive and inflexible Indeed a large part of the gameplay in many games is figuring out what the AI is programmed to do and learning to defeat it More recently machine learning techniques have begun to appear in video games This trend follows a long history of learning in board games originating from Samuel s 1959 checkers program that was based on a method similar to temporal difference learning Sutton 1988 followed by various learning methods applied to tic tac toe backgammon go othello and checkers see Fu rnkranz 2001 for a survey Many of these learning methods can be applied to video games as well For example Fogel et al 2004 trained teams of tanks and robots to fight each other
View Full Document
Unlocking...