Mitigating Reversal Curse in Large Language Models via Semantic-aware Permutation Training (2403.00758v3)

Abstract: While LLMs have achieved impressive performance across diverse tasks, recent studies showcase that causal LLMs suffer from the "reversal curse". It is a typical example that the model knows "A's father is B", but is unable to reason "B's child is A". This limitation poses a challenge to the advancement of artificial general intelligence (AGI), as it suggests a gap in the models' ability to comprehend and apply bidirectional reasoning. In this paper, we first conduct substantial evaluation and identify that the root cause of the reversal curse lies in the different word order between the training and inference stage, namely, the poor ability of causal LLMs to predict antecedent words within the training data. Accordingly, permutation on the training data is considered as a potential solution, since this can make the model predict antecedent words or tokens. However, previous permutation methods may disrupt complete phrases or entities, thereby posing challenges for the model to comprehend and learn from training data. To address this issue, we propose Semantic-aware Permutation Training (SPT), which addresses this issue by segmenting the training sentences into semantic units (i.e., entities or phrases) with an assistant LLM and permuting these units before feeding into the model. Extensive experiments demonstrate that SPT effectively mitigates the reversal curse since the performance on reversed questions approximates that on the forward ones, and significantly advances the performance of existing works.

• The paper introduces Semantic-aware Permutation Training (SPT) to mitigate the reversal curse by reorganizing semantic units in training.
• It segments sentences into meaningful chunks and applies three permutation orders—original, reversed, and random—to preserve semantic integrity.
• Extensive experiments show that SPT narrows the performance gap between forward and reversed queries, enhancing LLM reasoning towards AGI.

Mitigating the Reversal Curse via Semantic-aware Permutation Training

Introduction

The phenomenon known as the "reversal curse" reflects a significant challenge for causal LLMs, where these models struggle to infer reversed relational information accurately. For instance, while a model might correctly respond to "Who is A's father?" it often falters when asked the converse, "Who is B's child?" despite knowing that "A's father is B." This limitation not only highlights a gap in the LLM's understanding and reasoning capabilities but also poses a barrier to the progress towards AGI.

Evaluation and Analysis of the Reversal Curse

A comprehensive evaluation to pinpoint the root cause of the reversal curse reveals the fundamental issue lies in the model's limited capability to predict antecedent words or tokens, primarily due to the varying word order between training and inference stages. This discovery sets the stage for exploring potential solutions, highlighting the inadequacy of lightweight methods at inference to address this issue effectively.

Addressing the Reversal Curse with Semantic-aware Permutation Training (SPT)

In response to the identified challenges, this paper introduces Semantic-aware Permutation Training (SPT), an innovative approach to permutation training. SPT enhances the training process by segmenting sentences into semantic units or chunks, such as entities or phrases, and subsequently applying three distinct permutation orders: original, reversed, and randomly permuted. This method not only preserves semantic integrity during the permutation process but also significantly mitigates the reversal curse, as demonstrated through extensive experiments. The SPT method showcases superior performance over existing approaches, effectively narrowing the performance gap between forward and reversed questions.

Theoretical and Practical Implications

The proposed SPT method holds profound implications for both theory and practice in the field of LLMs and AGI. Theoretically, it sheds light on the underlying mechanisms responsible for the reversal curse and offers a novel perspective on enhancing LLMs' understanding capabilities through semantic-aware training methodologies. Practically, SPT paves the way for the development of more intelligent and versatile LLMs capable of navigating complex reasoning tasks with improved efficiency and accuracy. Such advancements are expected to contribute significantly towards the realization of AGI.

Future Directions

While the current paper marks a significant step forward in overcoming the reversal curse, it also opens up numerous avenues for future research. Exploring the full potential of bi-directional models in conjunction with SPT, for instance, could yield further improvements in models' understanding capabilities. Additionally, there's a need for more in-depth investigations into the broader implications of semantic-aware permutation strategies on the generative capabilities of LLMs across a wider range of tasks.

Conclusion

The Semantic-aware Permutation Training method introduced in this paper offers a promising solution to the challenge of the reversal curse in causal LLMs. By leveraging semantic segmentation and diverse permutation strategies, SPT significantly enhances models' ability to comprehend and reason through reversed relations, moving a step closer towards achieving AGI. This breakthrough not only demonstrates substantial progress in the field of LLMs but also lays a solid foundation for future innovations in model training methodologies.

Ethics and Limitations

The paper strictly adheres to ethical guidelines, employing publicly available datasets and ensuring fairness in evaluations. However, it acknowledges inherent limitations, including the reliance on pre-trained LLMs which may inherit biases from large internet-based training corpora. Future efforts are essential to continually refine these models, mitigate biases, and explore the broader societal impacts of advanced LLMs and their applications.