The use of gameplay data to inform high-level AI decision making
As game AI improves to the point where computer-controlled players prove to be worthy adversaries for humans, the topic broadens to include questions not just of AI strength, but also of fun: how interesting or engaging is the AI to play against? This is associated with the ability of the AI to exhibit “human-like” behaviour, although an “interesting” player is not necessarily “human-like,” just as a human player might not be interesting. However, “human-like” is a worthy goal for game AI: human players exhibit a variety of different styles dependent on their personal preferences, and often employ risky or suboptimal strategies within a game to express and to entertain themselves. Regardless, to design an opponent with strength as the only objective ignores the needs of the player, who requires both a challenge of the right level and an opponent that maintains their interest.
This project will investigate the use of gameplay data to inform high-level AI decision making, in an attempt to make existing game AI more human-like. This is related to the topic of player modelling, which involves using gameplay data to identify groups of players with similar play styles, in order to adapt a game to suit the player. Player types identified by this kind of analysis might be used as templates for different AI players, each making similar decisions to the corresponding human players when faced with similar situations.
This is not a straightforward learning task: one of the major challenges will lie in encoding the game state in terms of these recurring situations. For most computer games, the game’s own internal representation (i.e. the values of all of the variables required to describe the complete state of the game at any point) results in a prohibitively large state space. Much of this information is unnecessary, lying beyond what the player is able to perceive, but the bigger problem is that it is not a particularly useful representation: human players respond to highlevel behaviours such as “running for cover,” “setting up for an invasion,” or “surrounded,” and these are the kinds of features that the system must learn the appropriate responses for.
While such a representation could be created manually, by consultation with players or using knowledge of the game mechanics, a very interesting question is whether such a representation could be learned automatically. This will form the main part of the research, and has significance both inside and outside the field of game AI: a high-level representation of the game state sheds light on which aspects of the state are important to the player, and conversely on which aspects are redundant or useless. This could be used to inform, for example, the design of a game’s user interface, or future development of the game’s mechanics.
Using these representations, training data gathered from human gameplay would then be used to modify the action selection policies for AI players. The main questions here are to do with how (and if) this information can be used to improve the AI players. At this stage, it will be necessary to gather human responses in order to determine whether the new players are a significant improvement.
One or more simple games will be developed during the project, with which to gather data and perform the experiments. Additionally, working with games companies such as Team17 or Lionhead would provide the opportunity to obtain real-world data for complex commercial games. The first two stages of the project, game state representation and player modelling, can be performed using data from these commercial games, and the learnt representations in particular may be useful research outcomes for those companies. However, the final stage of the project requires modification of the game’s AI, to test the hypothesis that a modified opponent based on player responses would be more fun to play against. In general, commercial games do not allow this level of access; hence the need to develop these ‘in-house’ games.
Ultimately, this project aims to develop a useful method for learning high-level game state representations, based on real gameplay data. Of the many exciting uses for this type of representation, the project will focus on modifying game AI in order to create more interesting opponents and thereby improve the player’s experience
Adam is a keen programmer with a passion for games. Before starting his PhD, Adam studied physics at the University of York. After completing his MPhys, he moved to the University of Edinburgh for an MSc in artificial intelligence. Now back at York, he looks forward to applying his skills to advance the field of AI and to create better games.
Home institution: York
Ready to apply?
Once you have identified your potential supervisor, we would encourage you to contact them to discuss your research proposal.Learn More