Question Answering by Reasoning Across Documents with Graph Convolutional Networks (1808.09920v4)

Abstract: Most research in reading comprehension has focused on answering questions based on individual documents or even single paragraphs. We introduce a neural model which integrates and reasons relying on information spread within documents and across multiple documents. We frame it as an inference problem on a graph. Mentions of entities are nodes of this graph while edges encode relations between different mentions (e.g., within- and cross-document co-reference). Graph convolutional networks (GCNs) are applied to these graphs and trained to perform multi-step reasoning. Our Entity-GCN method is scalable and compact, and it achieves state-of-the-art results on a multi-document question answering dataset, WikiHop (Welbl et al., 2018).

• The paper introduces a GCN-based framework that aggregates entity mentions across documents to enable effective multi-hop reasoning.
• The paper replaces recurrent document encoders with pre-trained embeddings, reducing training time by up to fivefold while boosting efficiency.
• The paper achieves state-of-the-art results on the WikiHop dataset with over 2% accuracy improvement and an additional 3.6% gain using an ensemble model.

Reasoning Across Documents for Question Answering with Graph Convolutional Networks

The paper introduces a novel methodology for machine reading comprehension, focusing on complex question answering (QA) tasks that require reasoning across multiple documents. Traditional QA systems have primarily serviced single-document tasks with limited capacity for integrating disparate information. In response, the authors propose an innovative framework utilizing Graph Convolutional Networks (GCNs) to amalgamate and reason over connections across multiple documents, enhancing the interpretive capacity of QA systems.

Key Contributions

1. GCN-Based Entity Aggregation: The paper innovates by framing the QA task as an inference problem over a graph structure, where nodes represent entity mentions, and edges denote relationships such as coreference links. The proposed model leverages GCNs to propagate information across these graphs, thereby aligning with multi-hop reasoning requirements.
2. Scalability and Efficiency: By avoiding recurrent document encoders, which are computationally intensive, the model enhances efficiency. It utilizes pre-trained contextual embeddings like ELMo for node representation, reducing the computational overhead and enabling scalability to more extensive and complex document collections.
3. State-of-the-Art Results: Through empirical evaluation on the WikiHop dataset, the model establishes superior performance benchmarks, notably exceeding the results of existing solutions that primarily concatenate documents for reasoning.

Numerical and Empirical Insights

• The Entity-GCN model records a substantial performance improvement over previous models, achieving an over 2% increase in accuracy on benchmark datasets. Additionally, the ensemble model further elevates performance by approximately 3.6%.
• Training efficiency is highlighted by the model's faster processing capabilities compared to other prevalent models like BiDAF, showcasing a fivefold decrease in training time under equivalent settings.

Practical and Theoretical Implications

Practical applications are far-reaching, presenting a viable pathway for developing robust QA systems capable of tackling information synthesis from large-scale, heterogeneous document sources. Theoretically, the work underscores the potential of incorporating graph-based methods in NLP tasks that involve complex and multi-layered context comprehension.

Future Directions

The integration of graph-based methods within QA systems paves the way for further explorations into:

• Enhancing Coreference Resolution: Improving the reliability of node connections, particularly through more advanced coreference resolution mechanisms.
• Expanding Graph Relationships: Investigating more diverse relational embeddings within the graph to capture richer semantic interconnections among document entities.
• Real-World Challenges: Applying these methods to diverse domains, such as legal or medical document analysis, where nuanced, multi-document reasoning is critical.

The paper offers a transformative approach to multi-document QA, laying foundational strategies for future research in graph-based reasoning systems.