Real-Time Strategy Games: A New AI Research Challenge

1 Real-Time Strategy Games

Commercial computer games are a growing part of the en­ tertainment industry and simulations are a critical aspect of modern military training. The two fields have much in com­ mon, cross-fertilize, and are driving real-time AI research [Herz and Macedonia, 2002]. With the advent of fast per­ sonal computers, simulation-based games have become very popular. Today, these games constitute a multi-billion dollar enterprise. Examples are sports games and real-time strategy games. The common elements of simulation games are severe time constraints and a strong demand of real-time AI which must be capable of solving real-world decision tasks quickly and satisfactorily. Popular simulation games are therefore ideal test applications for real-time AI research.

Real-Time-Strategy (RTS) games - such as the million­ sellers Starcraft by Blizzard Entertainment and Age of Em­ pires by Ensemble Studios - can be viewed as simplified military simulations. Several players struggle over resources scattered over a 2D terrain by setting up an economy, build­ ing armies, and guiding them into battle in real-time. RTS games offer a large variety of fundamental AI research prob­ lems, unlike other game genres studied by the AI community so far:

• Resource management. Players start offby gathering local resources to build up defenses and attack forces, to upgrade weaponry, and to climb up the technology tree. Proper re­ source management is a vital part of any successful strategy.
• Decision making under uncertainty. Initially, players are not aware of the enemies' base locations and intentions. They have to gather intelligence by sending out scouts. If no infor­ mation is available yet, the players must form plausible hy­ potheses and act accordingly.
• Spatial and temporal reasoning. Static and dynamic ter­ rain analysis as well as understanding temporal relations of actions is of utmost importance in RTS games- and yet, cur­ rent game Als largely ignore these issues and fall victim to simple common-sense reasoning [Forbus et al., 2002].
• Collaboration. In RTS games groups of players can join forces and intelligence. How to coordinate actions effec­ tively by communication among the parties is a challenging research problem.
• Opponent modeling, Learning. One of the biggest short­ comings of most (RTS) game AI systems is their inability to learn from experience. Human players only need a couple of games to spot opponents' weaknesses and to exploit them in upcoming games. Current machine learning approaches in this area are inadequate.
• Adversarial real-time planning. In fine-grained simula­ tions, agents cannot afford to think in terms of micro actions. Instead, abstractions have to be found which allow a machine to conduct forward searches in a manageable abstract space and to translate found solutions back. Because the environ­ ment is also dynamic, hostile, and smart - adversarial real­ time planning approaches need to be investigated.

Playing RTS games is challenging. Even more challeng­ ing is the creation of autonomous real-time systems capable of outperforming human experts in this domain. Because search space abstraction, real-time planning, and temporal and spatial reasoning are central to many other problems, the scope of applications seems endless. One example is high­ performance combat simulators which are in large demand for training military personnel today and will become the core of automated battlefield decision-support systems of tomor­ row. [von der Lippe et al., 1999] predicts that 20% of the US armed forces will be robotic by 2015.

2 An RTS Game Programming Environment

The lack of AI interfaces even in upcoming RTS game titles makes it hard to conduct real-time AI research in this area and to compare the strength of the resulting AI systems with that of human experts. In order to solve this problem we launched an open source RTS game programming project [Buro, 2002] with the following goals:

• Building a hack-free server-client RTS game system. At the core of the system is a simulator to which players con­ nect via UNIX sockets (Fig. 1). The unique system features include: server-side simulation - which only sends visi­ ble information to clients thereby rendering common map­ revealing client hacks useless -and an open message proto­ col that allows AI researchers and players to connect what­ ever client software they like.
• Sparking competition among players and researchers. Popular games in which human players still have the upper hand are ideal test-domains for AI research. Unlike the con­ fined GUis of commercial RTS games, our open design al­ lows the construction of hybrid AI systems in which the hu­ man general is aided by AI modules of growing capabilities. Competitive game playing on an open Internet RTS game server is therefore likely to improve AI performance and er­ gonomic GUI design.
• Applying planning and machine learning techniques to RTS games. Classic game AI methods - such as alpha-

beta search - cannot be applied directly in this domain due to the large number of game objects, imperfect information, and real-time demands. Search space abstraction, hierarchi­ cal planning, and machine learning are approaches to over­ come these problems.

Our initial software release implemented a basic RTS pro­ gramming infrastructure . Recently, we added terrain fea­ tures (water, ground, plateaus, and ramps), view obstruction by terrain, and a split-view GUI which allows to watch differ­ ent parts of the playing field simultaneously. In order to main­ tain the high simulation speed, terrain and vision are based on tiles (Fig. 2). Soon, the complete source code of our RTS pro­ gramming environment will be released. It is being used in two thesis projects dealing with machine learning of low-level unit behavior and abstraction, planning, and heuristic search in RTS games. We hope to report first results in the RTS game AI domain later this year and to spark enough interest to be able to organize man-machine and machine-machine RTS game tournaments in the near future. For this it will be neces­ sary to define standard game setups including unit properties (size,speed,sight and attack ranges, ... ), technology tree, and resources. In contrast to commercial products, the focus here will be on simplicity to ease the initial AI development.

3 Related Work

The growing literature on AI in commercial games surely indicates the demand of smarter game AI. Many articles on planning are relevant to this project. Due to space limitation we only briefly mention J. Laird's Soar architecture and its application to first-person shooter games [Laird, 2001] and M. Atkin's work on the GRASP system that is applied to a capture-the-flag game [Atkin, 1998].

Computer soccer pursues similar goals and has become quite popular [Stone, 2002]. This domain can be regarded as a simplified RTS game with only a few objects, no economy, unremarkable terrain features, and more or less complete in-

RTS Game Server

formation. Another big difference is that in computer soccer a small number of agents are required to compute their ac­ tions autonomously whereas in RTS games a large number of objects have to be orchestrated.

The other body of literature relevant to this project is on military analyses and applications. Research in this area spans from mathematical combat models [Ilachinski, 1996] over computer generated forces - which are used in simula­ tion and training -to decision-support systems that aid com­ manders and troops on the battle-field or even control entire weapon systems autonomously. Our project tries to bring both research communities together.