Imitation Learning Datasets: A Toolkit For Creating Datasets, Training Agents and Benchmarking (2403.00550v1)

Abstract: Imitation learning field requires expert data to train agents in a task. Most often, this learning approach suffers from the absence of available data, which results in techniques being tested on its dataset. Creating datasets is a cumbersome process requiring researchers to train expert agents from scratch, record their interactions and test each benchmark method with newly created data. Moreover, creating new datasets for each new technique results in a lack of consistency in the evaluation process since each dataset can drastically vary in state and action distribution. In response, this work aims to address these issues by creating Imitation Learning Datasets, a toolkit that allows for: (i) curated expert policies with multithreaded support for faster dataset creation; (ii) readily available datasets and techniques with precise measurements; and (iii) sharing implementations of common imitation learning techniques. Demonstration link: https://nathangavenski.github.io/#/il-datasets-video

• The paper introduces a toolkit that enables rapid, multithreaded dataset generation and consistent benchmarking for imitation learning.
• It integrates curated and customizable expert policies to simplify expert data generation and enhance agent training through adaptable PyTorch structures.
• The approach ensures reproducible outcomes across diverse environments, lowering research barriers and advancing practical imitation learning applications.

A Comprehensive Overview of Imitation Learning Datasets Toolkit

Introduction to Imitation Learning Challenges

Imitation Learning (IL) has traditionally been hampered by the multifaceted challenge of generating and utilizing expert data for training agents. The iterative process of creating expert datasets for each new IL technique not only demands substantial time and resources but also introduces inconsistencies in performance evaluation across diverse IL approaches. This inconsistency is partly due to the variance in state and action distributions of each dataset. Furthermore, the quality and accessibility issues associated with existing datasets exacerbate the difficulty in benchmarking and developing IL techniques efficiently.

Addressing the Dataset Challenge

The work on Imitation Learning Datasets (IL-Datasets) serves as a potent response to the aforementioned challenges, presenting a toolkit designed to streamline the creation, utilization, and benchmarking of IL datasets. This toolkit is distinguished by its capacity for:

• Enabling rapid, multithreaded dataset generation leveraging curated expert policies, thus circumventing the need for prior expert training and reducing dataset creation discrepancies.
• Offering a repository of readily accessible datasets and facilitating the adaptation of these datasets to custom requirements, thereby accelerating the prototyping of new IL techniques.
• Providing a structured framework for IL technique benchmarking across a broad spectrum of environments, ensuring reproducible and consistent comparative analysis.

Simplifying Dataset Creation

At the core of the IL-Datasets toolkit is the multithreaded 'Controller' class, which empowers users to asynchronously capture 'Policy' experiences. This functionality not only ensures efficient use of computational resources but also supports the creation of datasets across various environments without the need for shared memory pointers. By integrating curated policies and offering the flexibility to incorporate custom policies, IL-Datasets significantly diminishes the divergence in behavior among different datasets. The process for dataset generation is streamlined to require minimal code, enhancing the accessibility of IL research.

Enhancing Training Efficiency

The 'BaselineDataset' class forms another critical component of the IL-Datasets toolkit, allowing researchers to incorporate both locally stored and HuggingFace-hosted datasets seamlessly into their work. This class, built upon the PyTorch Dataset structure, is designed for easy customization to accommodate diverse data formats, including sequential data. By simplifying access to large volumes of expert-generated episodes, IL-Datasets facilitates both the training and evaluation phases of IL agent development. This efficiency is further supported by detailed documentation of expert policies and performance metrics accompanying each dataset.

Benchmarking Innovations

A pivotal feature of IL-Datasets is its comprehensive benchmarking facility, which endeavors to apply and assess various IL techniques using the toolkit’s datasets. This effort aims to minimize the workload associated with IL technique development and lower the entry barriers for new researchers in the field. The benchmarking process is meticulously designed to ensure reproducibility, with specific training seeds employed to maintain consistency in training outcomes across different methods and environments. The provision of benchmarking results on the IL-Datasets platform facilitates an open and comparative analysis of IL techniques.

Conclusion and Future Directions

In summary, the IL-Datasets toolkit represents a significant stride toward resolving the pressing challenges associated with the creation and utilisation of IL datasets. By offering resources for rapid dataset generation, training assistance, and comprehensive benchmarking, IL-Datasets promises to enhance the consistency of IL research findings and foster an environment conducive to innovation. Looking ahead, the continued expansion of the IL-Datasets repository, coupled with advancements in IL techniques and training methodologies, is anticipated to drive further progress in IL research, bridging the gap between theoretical insight and practical application.

{acks} This research initiative has been supported by UK Research and Innovation, highlighting the collaborative effort in advancing the field of Safe and Trusted Artificial Intelligence.