The dynamics of higher-order novelties

Understanding how humans explore the world in search of novelties is key to foster innovation. Previous studies analyzed novelties in various exploration processes, defining them as the first appearance of an element. However, innovation can also be generated by novel association of what is already known. We hence define higher-order novelties as the first appearances of combinations of two or more elements, and we introduce higher-order Heaps' exponents as a way to characterize their pace of discovery. Through extensive analysis of real-world data, we find that processes with the same pace of discovery, as measured by the standard Heaps' exponent, can instead differ at higher orders. We then propose to model the dynamics of an exploration process as a random walk on an evolving network of the possible connections between elements. The model reproduces the empirical properties of higher-order novelties, revealing how the space of possibilities expands over time along with the exploration process.