NestE: Modeling Nested Relational Structures for Knowledge Graph Reasoning (2312.09219v1)

Abstract: Reasoning with knowledge graphs (KGs) has primarily focused on triple-shaped facts. Recent advancements have been explored to enhance the semantics of these facts by incorporating more potent representations, such as hyper-relational facts. However, these approaches are limited to \emph{atomic facts}, which describe a single piece of information. This paper extends beyond \emph{atomic facts} and delves into \emph{nested facts}, represented by quoted triples where subjects and objects are triples themselves (e.g., ((\emph{BarackObama}, \emph{holds_position}, \emph{President}), \emph{succeed_by}, (\emph{DonaldTrump}, \emph{holds_position}, \emph{President}))). These nested facts enable the expression of complex semantics like \emph{situations} over time and \emph{logical patterns} over entities and relations. In response, we introduce NestE, a novel KG embedding approach that captures the semantics of both atomic and nested factual knowledge. NestE represents each atomic fact as a $1\times3$ matrix, and each nested relation is modeled as a $3\times3$ matrix that rotates the $1\times3$ atomic fact matrix through matrix multiplication. Each element of the matrix is represented as a complex number in the generalized 4D hypercomplex space, including (spherical) quaternions, hyperbolic quaternions, and split-quaternions. Through thorough analysis, we demonstrate the embedding's efficacy in capturing diverse logical patterns over nested facts, surpassing the confines of first-order logic-like expressions. Our experimental results showcase NestE's significant performance gains over current baselines in triple prediction and conditional link prediction. The code and pre-trained models are open available at https://github.com/xiongbo010/NestE.

• The paper introduces NestE, a novel model that enhances KG embeddings by incorporating nested relational structures alongside atomic facts.
• It employs hypercomplex matrices and transformations to capture logical patterns and complex relations beyond traditional triple-based models.
• Experimental evaluations show significant gains in triple and conditional link prediction, advancing the state-of-the-art in knowledge graph reasoning.

Introduction

Knowledge graphs (KGs) serve as a crucial technology for organizing and querying relational data, depicting the connections among entities through facts, typically in the form of triples. These KGs have been instrumental in various applications, such as semantic search, recommendation systems, and question-answering utilities. To enable reasoning over KGs and prediction of missing links, KG embeddings have become increasingly popular. These models embed the entities and relationships of a graph into a lower-dimensional space while aiming to preserve the original graph's structural properties.

Beyond Triple Representations

Historically, KG embeddings have centered on individual triple-shaped facts, limiting their ability to capture more complex or contextualized information. To enhance this, recent research has incorporated more expressive data structures, such as hyper-relational facts. Nonetheless, these structures still address atomic facts and do not effectively capture the semantics where subjects and objects are related facts themselves, such as nested facts. The new approach, named NestE, builds upon this concept by focusing on the semantics of both atomic and nested facts in KGs, extending beyond the field of atomic fact representation.

NestE: A Novel Embedding Technique

NestE introduces an innovative modeling approach for KGs. It embeds atomic facts as hypercomplex matrices and nested relations as matrix transformations that operate on these atomic fact matrices through hypercomplex numbers. This intricate mathematical representation allows NestE to capture a diverse array of logical patterns and relations beyond first-order logic expressions. The representation is enhanced through a general hypercomplex framework that encompasses quaternions, hyperbolic quaternions, and split-quaternions. This variety of hypercomplex spaces provides a foundation for NestE to model structural patterns like hierarchies more effectively.

Evaluation and Performance

NestE demonstrates significant advancements over existing baseline models in tasks such as triple prediction and conditional link prediction, based on experimental evidence. The performance is evaluated against a variety of baselines and showcases the power of NestE's representation through substantial improvements in predictive tasks. Additionally, an ablation paper reveals the importance of nested fact embeddings in enhancing the model's performance. The use of hypercomplex embeddings, with their algebraic and geometric properties, paves the way for richer representations that are capable of inferring complex logical patterns for KG reasoning.

With the release of NestE, the KG community is provided with a flexible and robust framework for capturing and reasoning over nested relational structures. The inclusion of nested facts in modeling paves the way for representing and utilizing more expressive and contextual information from KGs, potentially leading to more intelligent AI systems capable of complex reasoning tasks.