Versatile Behavior Diffusion for Generalized Traffic Agent Simulation (2404.02524v2)

Abstract: Existing traffic simulation models often fail to capture the complexities of real-world scenarios, limiting the effective evaluation of autonomous driving systems. We introduce Versatile Behavior Diffusion (VBD), a novel traffic scenario generation framework that utilizes diffusion generative models to predict scene-consistent and controllable multi-agent interactions in closed-loop settings. VBD achieves state-of-the-art performance on the Waymo Sim Agents Benchmark and can effectively produce realistic and coherent traffic behaviors with complex agent interactions under diverse environmental conditions. Furthermore, VBD offers inference-time scenario editing through multi-step refinement guided by behavior priors and model-based optimization objectives. This capability allows for controllable multi-agent behavior generation, accommodating a wide range of user requirements across various traffic simulation applications. Despite being trained solely on publicly available datasets representing typical traffic conditions, we introduce conflict-prior and game-theoretic guidance approaches that enable the creation of interactive, long-tail safety-critical scenarios, which is essential for comprehensive testing and validation of autonomous vehicles. Lastly, we provide in-depth insights into effective training and inference strategies for diffusion-based traffic scenario generation models, highlighting best practices and common pitfalls. Our work significantly advances the ability to simulate complex traffic environments, offering a powerful tool for the development and assessment of autonomous driving technologies.