Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Enhancements for Real-Time Monte-Carlo Tree Search in General Video Game Playing (2407.03049v1)

Published 3 Jul 2024 in cs.AI

Abstract: General Video Game Playing (GVGP) is a field of Artificial Intelligence where agents play a variety of real-time video games that are unknown in advance. This limits the use of domain-specific heuristics. Monte-Carlo Tree Search (MCTS) is a search technique for game playing that does not rely on domain-specific knowledge. This paper discusses eight enhancements for MCTS in GVGP; Progressive History, N-Gram Selection Technique, Tree Reuse, Breadth-First Tree Initialization, Loss Avoidance, Novelty-Based Pruning, Knowledge-Based Evaluations, and Deterministic Game Detection. Some of these are known from existing literature, and are either extended or introduced in the context of GVGP, and some are novel enhancements for MCTS. Most enhancements are shown to provide statistically significant increases in win percentages when applied individually. When combined, they increase the average win percentage over sixty different games from 31.0% to 48.4% in comparison to a vanilla MCTS implementation, approaching a level that is competitive with the best agents of the GVG-AI competition in 2015.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (29)
  1. J. Levine, C. B. Congdon, M. Ebner, G. Kendall, S. M. Lucas, R. M. anad T. Schaul, and T. Thompson, “General Video Game Playing,” in Artif. and Comput. Intell. in Games, ser. Dagstuhl Follow-Ups, S. M. Lucas, M. Mateas, M. Preuss, P. Spronck, and J. Togelius, Eds.   Schloss Dagstuhl–Leibniz-Zentrum fuer Informatik, 2013, vol. 6, pp. 77–83.
  2. M. Genesereth, N. Love, and B. Pell, “General Game Playing: Overview of the AAAI Competition,” AI Magazine, vol. 26, no. 2, pp. 62–72, 2005.
  3. M. G. Bellemare, Y. Naddaf, J. Veness, and M. Bowling, “The Arcade Learning Environment: An Evaluation Platform for General Agents,” Journal of Artif. Intell. Research, vol. 47, pp. 253–279, 2013.
  4. D. Perez, S. Samothrakis, J. Togelius, T. Schaul, S. M. Lucas, A. Couëtoux, J. Lee, C.-U. Lim, and T. Thompson, “The 2014 General Video Game Playing Competition,” IEEE Trans. on Comput. Intell. and AI in Games, 2016, To appear.
  5. M. Ebner, J. Levine, S. M. Lucas, T. Schaul, T. Thompson, and J. Togelius, “Towards a Video Game Description Language,” in Artif. and Comput. Intell. in Games, ser. Dagstuhl Follow-Ups, S. M. Lucas, M. Mateas, M. Preuss, P. Spronck, and J. Togelius, Eds.   Schloss Dagstuhl–Leibniz-Zentrum fuer Informatik, 2013, vol. 6, pp. 85–100.
  6. T. Schaul, “A Video Game Description Language for Model-based or Interactive Learning,” in Proc. of the IEEE Conf. on Comput. Intell. in Games.   Niagara Falls: IEEE Press, 2013, pp. 193–200.
  7. D. Perez-Liebana, S. Samothrakis, J. Togelius, S. M. Lucas, and T. Schaul, “General Video Game AI: Competition, Challenges and Opportunities,” in Proc. of the Thirtieth AAAI Conf. on Artif. Intell.   AAAI Press, 2016, pp. 4335–4337.
  8. L. Kocsis and C. Szepesvári, “Bandit Based Monte-Carlo Planning,” in Machine Learning: ECML 2006, ser. LNCS, J. Fürnkranz, T. Scheffer, and M. Spiliopoulou, Eds.   Springer Berlin Heidelberg, 2006, vol. 4212, pp. 282–293.
  9. R. Coulom, “Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search,” in Computers and Games, ser. LNCS, H. J. van den Herik, P. Ciancarini, and H. H. L. M. Donkers, Eds., vol. 4630.   Springer Berlin Heidelberg, 2007, pp. 72–83.
  10. Y. Björnsson and H. Finnsson, “CadiaPlayer: A Simulation-Based General Game Player,” IEEE Trans. on Comput. Intell. and AI in Games, vol. 1, no. 1, pp. 4–15, 2009.
  11. D. Perez, J. Dieskau, M. Hünermund, S. Mostaghim, and S. M. Lucas, “Open Loop Search for General Video Game Playing,” in Proc. of the Genetic and Evol. Computation Conf.   ACM, 2015, pp. 337–344.
  12. G. M. J.-B. Chaslot, M. H. M. Winands, H. J. van den Herik, J. W. H. M. Uiterwijk, and B. Bouzy, “Progressive Strategies for Monte-Carlo Tree Search,” New Mathematics and Natural Computation, vol. 4, no. 3, pp. 343–357, 2008.
  13. P. Auer, N. Cesa-Bianchi, and P. Fischer, “Finite-time Analysis of the Multiarmed Bandit Problem,” Machine Learning, vol. 47, no. 2-3, pp. 235–256, 2002.
  14. C. Browne, E. Powley, D. Whitehouse, S. Lucas, P. I. Cowling, P. Rohlfshagen, S. Tavener, D. Perez, S. Samothrakis, and S. Colton, “A Survey of Monte Carlo Tree Search Methods,” IEEE Trans. on Comput. Intell. and AI in Games, vol. 4, no. 1, pp. 1–43, 2012.
  15. J. A. M. Nijssen and M. H. M. Winands, “Enhancements for Multi-Player Monte-Carlo Tree Search,” in Computers and Games (CG 2010), ser. LNCS, H. J. van den Herik, H. Iida, and A. Plaat, Eds.   Springer Berlin Heidelberg, 2011, vol. 6515, pp. 238–249.
  16. M. J. W. Tak, M. H. M. Winands, and Y. Björnsson, “N-Grams and the Last-Good-Reply Policy Applied in General Game Playing,” IEEE Trans. on Comput. Intell. and AI in Games, vol. 4, no. 2, pp. 73–83, 2012.
  17. T. Pepels, M. H. M. Winands, and M. Lanctot, “Real-Time Monte Carlo Tree Search in Ms Pac-Man,” IEEE Trans. on Comput. Intell. and AI in Games, vol. 6, no. 3, pp. 245–257, 2014.
  18. T. Geffner and H. Geffner, “Width-Based Planning for General Video-Game Playing,” in Proc. of the Eleventh Artif. Intell. and Interactive Digital Entertainment International Conf., A. Jhala and N. Sturtevant, Eds.   AAAI Press, 2015, pp. 23–29.
  19. R. Ramanujan, A. Sabharwal, and B. Selman, “On Adversarial Search Spaces and Sampling-Based Planning,” in 20th International Conf. on Automated Planning and Scheduling, R. I. Brafman, H. Geffner, J. Hoffmann, and H. A. Kautz, Eds.   AAAI, 2010, pp. 242–245.
  20. H. Finnsson and Y. Björnsson, “Game-Tree Properties and MCTS Performance,” in IJCAI’11 Workshop on General Intelligence in Game Playing Agents (GIGA’11), Y. Björnsson, N. Sturtevant, and M. Thielscher, Eds., 2011, pp. 23–30.
  21. H. Baier and M. H. M. Winands, “MCTS-Minimax Hybrids,” IEEE Trans. on Comput. Intell. and AI in Games, vol. 7, no. 2, pp. 167–179, 2015.
  22. N. Lipovetzky and H. Geffner, “Width and Serialization of Classical Planning Problems,” in Proc. of the Twentieth European Conf. on Artif. Intell. (ECAI 2012), L. De Raedt, C. Bessiere, D. Dubois, P. Doherty, P. Frasconi, F. Heintz, and P. Lucas, Eds.   IOS Press, 2012, pp. 540–545.
  23. D. Perez, S. Samothrakis, and S. Lucas, “Knowledge-based Fast Evolutionary MCTS for General Video Game Playing,” in Proc. of the IEEE Conf. on Comput. Intell. and Games.   IEEE, 2014, pp. 68–75.
  24. J. van Eeden, “Analysing and Improving the Knowledge-based Fast Evolutionary MCTS Algorithm,” Master’s thesis, Utrecht University, Utrecht, the Netherlands, 2015.
  25. C. Y. Chu, H. Hashizume, Z. Guo, T. Harada, and R. Thawonmas, “Combining Pathfinding Algorithm with Knowledge-based Monte-Carlo Tree Search in General Video Game Playing,” in Proc. of the IEEE Conf. on Comput. Intell. and Games.   IEEE, 2015, pp. 523–529.
  26. P. E. Hart, N. J. Nilsson, and B. Raphael, “A Formal Basis for the Heuristic Determination of Minimum Cost Paths,” Systems Science and Cybernetics, IEEE Transactions on, vol. 4, no. 2, pp. 100–107, 1968.
  27. E. J. Jacobsen, R. Greve, and J. Togelius, “Monte Mario: Platforming with MCTS,” in Proc. of the 2014 Conf. on Genetic and Evolutionary Computation.   ACM, 2014, pp. 293–300.
  28. F. Frydenberg, K. R. Andersen, S. Risi, and J. Togelius, “Investigating MCTS Modifications in General Video Game Playing,” in Proc. of the IEEE Conf. on Comput. Intell. and Games.   IEEE, 2015, pp. 107–113.
  29. T. Schuster, “MCTS Based Agent for General Video Games,” Master’s thesis, Department of Knowledge Engineering, Maastricht University, Maastricht, the Netherlands, 2015.
Citations (45)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now