BEACON: A Bayesian Optimization Strategy for Novelty Search in Expensive Black-Box Systems (2406.03616v2)

Abstract: Novelty search (NS) refers to a class of exploration algorithms that automatically uncover diverse system behaviors through simulations or experiments. Uncovering diversity is a key aspect of engineering design problems with connections to material and drug discovery, neural architecture search, reinforcement learning, and robot navigation. Since the relationship between the inputs and behaviors (outputs) of modern engineering systems not always available or easily represented in closed analytical form, novelty search must be able to handle model opacity. For systems whose behaviors are expensive to simulate or evaluate, we propose a sample-efficient NS method inspired by Bayesian optimization principles. This involves modeling the input-to-behavior mapping with multi-output Gaussian processes (MOGP) and selecting inputs to evaluate that maximize a novelty metric while balancing the exploration-exploitation trade-off. By leveraging advances in efficient posterior sampling and high-dimensional Gaussian process modeling, we discuss how our approach can be made scalable with respect to both the amount of data and number of inputs. We demonstrate the potential of our approach on several well-studied benchmark problems and multiple real-world examples. We show that BEACON comprehensively outperforms existing baselines by finding substantially larger sets of diverse behaviors under limited sampling budgets.