Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
158 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Exploring Parity Challenges in Reinforcement Learning through Curriculum Learning with Noisy Labels (2312.05379v2)

Published 8 Dec 2023 in cs.AI

Abstract: This paper delves into applying reinforcement learning (RL) in strategy games, particularly those characterized by parity challenges, as seen in specific positions of Go and Chess and a broader range of impartial games. We propose a simulated learning process, structured within a curriculum learning framework and augmented with noisy labels, to mirror the intricacies of self-play learning scenarios. This approach thoroughly analyses how neural networks (NNs) adapt and evolve from elementary to increasingly complex game positions. Our empirical research indicates that even minimal label noise can significantly impede NNs' ability to discern effective strategies, a difficulty that intensifies with the growing complexity of the game positions. These findings underscore the urgent need for advanced methodologies in RL training, specifically tailored to counter the obstacles imposed by noisy evaluations. The development of such methodologies is crucial not only for enhancing NN proficiency in strategy games with significant parity elements but also for broadening the resilience and efficiency of RL systems across diverse and complex environments.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (30)
  1. Provable advantage of curriculum learning on parity targets with mixed inputs. arXiv preprint arXiv:2306.16921, 2023.
  2. Mohammed Al-Rawi. A neural network to solve the hybrid n-parity: Learning with generalization issues. Neurocomputing, 68:273–280, 2005.
  3. Pondernet: Learning to ponder. arXiv preprint arXiv:2107.05407, 2021.
  4. Winning ways for your mathematical plays, volume 1. AK Peters/CRC Press, 2001.
  5. Tom Bylander. Learning linear threshold functions in the presence of classification noise. In Proceedings of the seventh annual conference on Computational learning theory, pages 340–347, 1994.
  6. Regularizing towards permutation invariance in recurrent models. arXiv preprint arXiv:2010.13055, 2020.
  7. A mathematical model for curriculum learning for parities. 2023.
  8. Learning parities with neural networks. Advances in Neural Information Processing Systems, 33, 2020.
  9. Reinforcement learning with a corrupted reward channel. arXiv preprint arXiv:1705.08417, 2017.
  10. Generalization properties of modular networks: implementing the parity function. IEEE transactions on neural networks, 12(6):1306–1313, 2001.
  11. Michael Hahn. Theoretical limitations of self-attention in neural sequence models. Transactions of the Association for Computational Linguistics, 8:156–171, 2020.
  12. Long short-term memory. Neural computation, 9(8):1735–1780, 1997.
  13. Solving the n-bit parity problem using neural networks. Neural Networks, 12(9):1321–1323, 1999.
  14. N-bit parity neural networks: new solutions based on linear programming. Neurocomputing, 48(1-4):477–488, 2002.
  15. Peer loss functions: Learning from noisy labels without knowing noise rates. In International conference on machine learning, pages 6226–6236. PMLR, 2020.
  16. Learning with noisy labels. Advances in neural information processing systems, 26, 2013.
  17. Bostrom Nick. Superintelligence: Paths, dangers, strategies. 2014.
  18. Clayton Scott. A rate of convergence for mixture proportion estimation, with application to learning from noisy labels. In Artificial Intelligence and Statistics, pages 838–846. PMLR, 2015.
  19. Classification with asymmetric label noise: Consistency and maximal denoising. In Conference on learning theory, pages 489–511. PMLR, 2013.
  20. Failures of gradient-based deep learning. In International Conference on Machine Learning, pages 3067–3075. PMLR, 2017.
  21. On the computational power of neural nets. Journal of computer and system sciences, 50(1):132–150, 1995.
  22. Mastering the game of go with deep neural networks and tree search. nature, 529(7587):484–489, 2016.
  23. Mastering the game of go without human knowledge. nature, 550(7676):354–359, 2017.
  24. A general reinforcement learning algorithm that masters chess, shogi, and go through self-play. Science, 362(6419):1140–1144, 2018.
  25. Learning from noisy labels with deep neural networks: A survey. IEEE Transactions on Neural Networks and Learning Systems, 2022.
  26. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008, 2017.
  27. Reinforcement learning with perturbed rewards. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 6202–6209, 2020.
  28. Solving parity-n problems with feedforward neural networks. In Proceedings of the International Joint Conference on Neural Networks, 2003., volume 4, pages 2546–2551. IEEE, 2003.
  29. Do rnn and lstm have long memory? In International Conference on Machine Learning, pages 11365–11375. PMLR, 2020.
  30. Impartial games: A challenge for reinforcement learning. arXiv preprint arXiv:2205.12787, 2022.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now