2000 character limit reached
ScriptWorld: Text Based Environment For Learning Procedural Knowledge (2307.03906v1)
Published 8 Jul 2023 in cs.CL, cs.AI, cs.LG, and cs.MA
Abstract: Text-based games provide a framework for developing natural language understanding and commonsense knowledge about the world in reinforcement learning based agents. Existing text-based environments often rely on fictional situations and characters to create a gaming framework and are far from real-world scenarios. In this paper, we introduce ScriptWorld: a text-based environment for teaching agents about real-world daily chores and hence imparting commonsense knowledge. To the best of our knowledge, it is the first interactive text-based gaming framework that consists of daily real-world human activities designed using scripts dataset. We provide gaming environments for 10 daily activities and perform a detailed analysis of the proposed environment. We develop RL-based baseline models/agents to play the games in Scriptworld. To understand the role of LLMs in such environments, we leverage features obtained from pre-trained LLMs in the RL agents. Our experiments show that prior knowledge obtained from a pre-trained LLM helps to solve real-world text-based gaming environments. We release the environment via Github: https://github.com/Exploration-Lab/ScriptWorld
- Learning Dynamic Belief Graphs to Generalize on Text-Based Games. In NeurIPS, 2020.
- LeDeepChef: Deep Reinforcement Learning Agent for Families of Text-Based Games. In AAAI, 2020.
- Graph Constrained Reinforcement Learning for Natural Language Action Spaces. In International Conference on Learning Representations, 2020.
- Transfer in Deep Reinforcement Learning Using Knowledge Graphs. In Proceedings of the Thirteenth Workshop on Graph-Based Methods for Natural Language Processing (TextGraphs-13), 2019.
- Multiple Sequence Alignment Modeling: Methods and Applications. Briefings in Bioinformatics, 2016.
- Bootstrapped Q-learning with Context Relevant Observation Pruning to Generalize in Text-based Games. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020.
- TextWorld: A Learning Environment for Text-based Games. In CGW@IJCAI, 2018.
- A Hierarchical Bayesian Model for Unsupervised Induction of Script Knowledge. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, 2014.
- Interactive Fiction Games: A Colossal Adventure. In AAAI, 2020.
- Deep Reinforcement Learning with a Natural Language Action Space. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016.
- Understanding Grounded Language Learning Agents. arXiv preprint arXiv:1710.09867, 2017.
- Skip N-grams and Ranking Functions for Predicting Script Events. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, 2012.
- Planning and Acting in Partially Observable Stochastic Domains. Artificial Intelligence, 1998.
- The NetHack Learning Environment. In NeurIPS, 2020.
- Subsymbolic Natural Language Processing: An Integrated Model of Scripts, Lexicon, and Memory. MIT press, 1993.
- Playing Atari with Deep Reinforcement Learning. arXiv preprint arXiv:1312.5602, 2013.
- Asynchronous Methods for Deep Reinforcement Learning. In International Conference on Machine Learning, 2016.
- Inducing Neural Models of Script Knowledge. In Proceedings of the Eighteenth Conference on Computational Natural Language Learning, 2014.
- InScript: Narrative texts annotated with script information. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), 2016.
- Modeling Semantic Expectation: Using Script Knowledge for Referent Prediction. Transactions of the Association for Computational Linguistics, 2017.
- Ashutosh Modi. Event Embeddings for Semantic Script Modeling. In Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning, 2016.
- Ashutosh Modi. Modeling Common Sense Knowledge via Scripts. PhD thesis, Saarland University, 2017.
- Text-based RL Agents with Commonsense Knowledge: New Challenges, Environments and Baselines. In AAAI Conference on Artificial Intelligence, 2020.
- Language Understanding for Text-based Games using Deep Reinforcement Learning. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 2015.
- Toward Understanding Catastrophic Forgetting in Continual Learning. arXiv preprint arXiv:1908.01091, abs/1908.01091, 2019.
- MCScript: A Novel Dataset for Assessing Machine Comprehension Using Script Knowledge. In LREC, 2018.
- SemEval-2018 Task 11: Machine Comprehension Using Commonsense Knowledge. In Proceedings of the 12th International Workshop on Semantic Evaluation, 2018.
- GloVe: Global Vectors for Word Representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014.
- Using Sentence-Level LSTM Language Models for Script Inference. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016.
- Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.
- Learning Script Knowledge with Web Experiments. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, 2010.
- Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019.
- Learning to predict script events from domain-specific text. In Proceedings of the Fourth Joint Conference on Lexical and Computational Semantics, 2015.
- proScript: Partially Ordered Scripts Generation. In Findings of the Association for Computational Linguistics: EMNLP, 2021.
- What do Large Language Models Learn about Scripts? In Proceedings of the 11th Joint Conference on Lexical and Computational Semantics, 2022.
- Scripts, Plans, and Knowledge. In Proceedings of the 4th International Joint Conference on Artificial Intelligence, IJCAI, 1975.
- Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.
- Pre-trained Language Models as Prior Knowledge for Playing Text-based Games. In 21st International Conference on Autonomous Agents and Multiagent Systems, AAMAS, 2022.
- Reinforcement Learning: An Introduction. MIT press, 2018.
- A Crowdsourced Database of Event Sequence Descriptions for the Acquisition of High-quality Script Knowledge. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), 2016.
- Keep CALM and Explore: Language Models for Action Generation in Text-based Games. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020.
- Learn how to cook a new recipe in a new house: Using map familiarization, curriculum learning, and bandit feedback to learn families of text-based adventure games. arXiv preprint arXiv:1908.04777, 2019.
- Interactive Language Learning by Question Answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019.
Collections
Sign up for free to add this paper to one or more collections.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
