JORLDY: a fully customizable open source framework for reinforcement learning (2204.04892v1)

Abstract: Recently, Reinforcement Learning (RL) has been actively researched in both academic and industrial fields. However, there exist only a few RL frameworks which are developed for researchers or students who want to study RL. In response, we propose an open-source RL framework "Join Our Reinforcement Learning framework for Developing Yours" (JORLDY). JORLDY provides more than 20 widely used RL algorithms which are implemented with Pytorch. Also, JORLDY supports multiple RL environments which include OpenAI gym, Unity ML-Agents, Mujoco, Super Mario Bros and Procgen. Moreover, the algorithmic components such as agent, network, environment can be freely customized, so that the users can easily modify and append algorithmic components. We expect that JORLDY will support various RL research and contribute further advance the field of RL. The source code of JORLDY is provided on the following Github: https://github.com/kakaoenterprise/JORLDY

• The paper presents JORLDY as a fully customizable open-source framework integrating over 20 RL algorithms with support for multiple environments like OpenAI Gym and Mujoco.
• Its modular architecture allows independent modification of agents, network structures, and optimization strategies to foster innovative RL experiments.
• The framework supports efficient distributed training via Ray, providing scalable benchmarking across varied RL tasks and applications.

JORLDY: A Comprehensive Framework for Reinforcement Learning

The paper presents "JORLDY", an open-source framework designed to address the increasing need for customizable and user-friendly tools in reinforcement learning (RL). Developed by researchers at Kakao Enterprise, JORLDY aims to facilitate RL research by providing a versatile environment that supports a wide array of algorithms and applications.

Overview of JORLDY

JORLDY comprises over 20 well-established RL algorithms, implemented using PyTorch, and supports multiple RL environments such as OpenAI Gym, Unity ML-Agents, Mujoco, Super Mario Bros, and Procgen. One of its core strengths is its flexibility, allowing users to readily modify and extend algorithmic components such as agents, networks, and environments. This adaptability makes JORLDY a potent tool for both academic research and industrial applications.

Key Features

Customizability and Modularity

JORLDY provides a modular architecture where individual components, including agent architectures, network structures, and optimization strategies, can be independently customized or combined. This design caters to researchers aiming to experiment with new ideas without being encumbered by restrictive frameworks. For example, users can easily interchange components between algorithms to create hybrids, such as incorporating IQN into Rainbow-DQN.

Support for Distributed Training

The framework supports distributed training through integration with Ray, offering users the choice between synchronous and asynchronous distributed setups. This feature allows for substantial scalability and efficiency in training RL models, aligning with modern advancements in RL research where distributed training is crucial for achieving state-of-the-art performance.

Comprehensive Algorithm and Environment Support

JORLDY includes a variety of RL algorithms categorized into value-based, policy-based, actor-critic, distributional RL, exploration-focused, and distributed structure algorithms. These are implemented with a focus on ease of use and performance benchmarking across diverse environments. Its broad environment compatibility means that researchers can leverage JORLDY to apply RL beyond traditional gaming contexts, extending to sectors like robotics and industrial automation.

Benchmarking Results

The paper provides detailed benchmarks across prominent environments like Atari and MuJoCo, highlighting JORLDY’s performance in various settings. The results demonstrate competitive scores, with the framework showing robust capabilities in standard RL tasks. This benchmarking serves as a baseline for researchers looking to evaluate or compare new methodologies using JORLDY.

Implications and Future Directions

The development of JORLDY has practical and theoretical implications. Practically, it lowers the barrier to entry for non-experts interested in deploying RL solutions, while simultaneously facilitating advanced research by providing a flexible experimentation platform. Theoretically, JORLDY’s design encourages exploration of novel RL architectures and strategies, potentially propelling new advancements in the field.

Looking forward, continuous development and community engagement with JORLDY could lead to enhancements in both its capabilities and user-friendliness, fostering further breakthroughs in RL research and application. As the RL landscape evolves, frameworks like JORLDY will play a pivotal role in shaping the future of intelligent systems design.

In summary, the paper introduces JORLDY as a significant contribution to the RL community, offering a practical and versatile toolkit for researchers and developers alike. Its open-source nature, backed by comprehensive algorithmic and environment support, positions it as a valuable resource for ongoing and future RL initiatives.