Interactive Fiction Games: A Colossal Adventure (1909.05398v3)

Abstract: A haLLMark of human intelligence is the ability to understand and communicate with language. Interactive Fiction games are fully text-based simulation environments where a player issues text commands to effect change in the environment and progress through the story. We argue that IF games are an excellent testbed for studying language-based autonomous agents. In particular, IF games combine challenges of combinatorial action spaces, language understanding, and commonsense reasoning. To facilitate rapid development of language-based agents, we introduce Jericho, a learning environment for man-made IF games and conduct a comprehensive study of text-agents across a rich set of games, highlighting directions in which agents can improve.

• The paper introduces Jericho, a platform that simplifies connecting reinforcement learning agents to interactive fiction games via game-specific vocabulary and action templates.
• It employs a template-based action space that drastically reduces the combinatorial search space for generating natural language commands.
• Empirical evaluations with agents like DRRN and TDQN reveal both promising performance metrics and ongoing challenges in language understanding and sequential decision-making.

Evaluating Language-Based Agents in Interactive Fiction Games

The paper explores the utilization of Interactive Fiction (IF) games as a testbed for the development and evaluation of language-based autonomous agents. IF games are text-based simulation environments where players navigate and interact solely through textual input, which the authors argue presents a set of unique challenges suitable for studying language understanding, commonsense reasoning, and sequential decision making in artificial agents. The development of Jericho, an open-source Python-based platform that facilitates the connection of reinforcement learning agents with IF games, allows for an empirical evaluation of such agents across a range of games.

Contributions and Methodology

This research makes three primary contributions:

1. Introduction of Jericho: Jericho provides a Gym-like interface for connecting reinforcement learning agents to IF games, supporting features such as loading/saving game states and replicating gameplay through seeding. This framework includes mechanisms for extracting game-specific vocabulary and action templates, which significantly reduces the complexity of language generation required from agents.
2. Template-Based Action Space: The authors propose a template-based action space for action generation. This involves selecting action templates and subsequently populating them with words from the game-specific vocabulary. This approach drastically reduces the effective search space from potentially billions of combinations to manageable millions.
3. Empirical Evaluation of Agents: The paper evaluates several agents, including DRRN for choice-based games, and TDQN, which builds on the LSTM-DQN by operating in the novel template-based action space. The evaluation spans across a robust set of human-made IF games, providing quantitative benchmarks for agent performance.

Research Challenges

Several challenges in applying reinforcement learning to IF games are discussed:

• Combinatorial Action Space: The vast number of potential natural language inputs presents a significant exploration challenge. Even with a constrained action space, agents must efficiently explore millions of possibilities to learn effective strategies.
• Commonsense Reasoning: IF games often rely on an understanding of ungrounded language and require agents to leverage commonsense reasoning for effective interaction and decision making, a task that remains largely unsolved in AI.
• Knowledge Representation: IF games necessitate complex tracking of state information across numerous locations and objects. Successfully mapping navigational experience and item interactions to an internal representation poses a challenge for AI agents.

Results and Implications

The empirical results underscore the viability of reinforcement learning within IF games, demonstrating variable performance across games of differing complexity. DRRN outperforms TDQN, indicating the utility of pre-identified valid actions in choice-based settings. However, TDQN showcases the potential of language generation tasks even within the complexity and variability inherent to IF game environments.

The success and limitations of the explored agents indicate significant room for future exploration, particularly in developing general game-playing agents capable of transferring learned policies across different IF environments. Moreover, this research prompts consideration of unsupervised learning strategies and improved methods for action space exploration.

Conclusion

The work positions IF games as a critical environment for advancing AI research in natural language understanding and decision making. The introduction of Jericho and the template-based action space constitute valuable contributions to the field, offering an extensible platform for subsequent investigation. As IF games continue to challenge both human players and AI, they represent a fertile ground for future advancements in language-based agent research.