- The paper introduces a novel AI agent that uses 5.5 million frames of human gameplay to achieve a humanlike performance in CSGO deathmatch.
- It employs an EfficientNetB0 model combined with convolutional LSTM layers and a discretized action space to efficiently process reduced-resolution pixel inputs.
- The approach matches medium-difficulty in-game AI performance and establishes a new benchmark, opening avenues for future research in complex FPS environments.
Insights into Counter-Strike Deathmatch with Large-Scale Behavioural Cloning
This paper introduces an AI agent designed to play the popular first-person-shooter (FPS) game Counter-Strike: Global Offensive (CSGO) using pixel-based input. The agent utilizes a deep neural network and employs a novel approach based on large-scale behavioral cloning. Previous gaming AI research has been focused primarily on games with accessible APIs and low-resolution graphics, such as Doom and VizDoom, allowing these games to be simulated at high speeds and low computational costs. This paper shifts the focus to CSGO, a game with substantially higher computational requirements, posing a significant challenge for data generation using traditional reinforcement learning techniques.
Methodology and Implementation
The paper's main methodology is behavioral cloning, which benefits from a large dataset comprising human gameplay data collected from public servers and expert demonstrations. The dataset contains approximately 5.5 million frames, equivalent to 95 hours of gameplay, which to the best of the authors' knowledge is one of the largest datasets used in imitation learning for FPS games.
This two-stage approach, which starts with training the agent on a broad and noisy dataset followed by fine-tuning on a smaller purified dataset of expert demonstrations, addresses the limitations seen in prior work where limited data led to underperforming agents. The trained agent matches the medium-difficulty AI built into CSGO and exhibits humanlike gameplay behavior, with a performance level akin to casual human players.
Technical Design
Regarding the agent's architectural design, the game visual inputs are processed at a resolution significantly reduced from the original, allowing them to be compatible with available GPU resources. The agent's architecture consists of an EfficientNetB0 model in conjunction with convolutional LSTM layers to handle temporal dependencies, and independent binary cross-entropy losses for action prediction. Notably, the action space is partially discretized to enhance performance, especially in tasks requiring precise aiming, which is vital in FPS games.
Results and Contribution
The agent's evaluation includes comparison against human players of varying proficiency and CSGO's built-in AI. While the agent underperforms relative to expert human players, its approaches mimic human behavior closely, especially in deathmatch modes. This humanlike behavior is evaluated using qualitative observations and quantitative assessments, such as map coverage heatmaps, which illustrate that the agent's movement patterns align closely with human players more than the built-in AI bots.
As a pioneering effort in its domain, this work provides an important contribution to the AI research community by proposing a method that can potentially generalize to other modern video games. Furthermore, the paper opens new avenues for future research, especially concerning the evolving field of offline reinforcement learning, where leveraging large datasets of existing gameplay is crucial.
Speculation on Future Directions
Looking forward, the methods could be refined by integrating advanced imitation learning or reward-based learning approaches, potentially improving the agent's competitiveness in more complex game scenarios such as CSGO's full competitive mode. Additionally, further exploration into zero-shot generalization to new maps provides promising opportunities to develop more adaptable gaming AI systems.
The paper offers not only an effective framework for building data- and compute-efficient agents in complex environments but also presents a valuable new benchmark for AI researchers interested in FPS games. The open-source distribution of the developed code and dataset is expected to inspire subsequent research, encouraging further advancements across the intersection of AI and complex multi-agent systems inherent in modern video games.