Test Framework for a Virtual Competition Testbed

Abstract: Virtual environments have been utilised in robotics research as a tool to assess systems before deploying them in the field. The COVID-19 pandemic has brought about additional motivation for the development of virtual benchmarks in order to aid in safe and productive development. In-person robotics competitions have also halted, thus limiting the scope of opportunities for students and researchers. We implemented the structure of a service robotics competition into an extendable and adaptable virtual scoring environment. The competition challenges the state of the art in home service robotics by presenting realistic household tasks for robots to complete. The virtual environment provides a foundation for competition teams to assess their systems when accessing the physical environment is not possible. We believe that utilising virtual environments as a means of assessment will lead to other benefits such as increased access and generalisation.