RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models (2401.00396v2)

Abstract: Retrieval-augmented generation (RAG) has become a main technique for alleviating hallucinations in LLMs. Despite the integration of RAG, LLMs may still present unsupported or contradictory claims to the retrieved contents. In order to develop effective hallucination prevention strategies under RAG, it is important to create benchmark datasets that can measure the extent of hallucination. This paper presents RAGTruth, a corpus tailored for analyzing word-level hallucinations in various domains and tasks within the standard RAG frameworks for LLM applications. RAGTruth comprises nearly 18,000 naturally generated responses from diverse LLMs using RAG. These responses have undergone meticulous manual annotations at both the individual cases and word levels, incorporating evaluations of hallucination intensity. We not only benchmark hallucination frequencies across different LLMs, but also critically assess the effectiveness of several existing hallucination detection methodologies. Furthermore, we show that using a high-quality dataset such as RAGTruth, it is possible to finetune a relatively small LLM and achieve a competitive level of performance in hallucination detection when compared to the existing prompt-based approaches using state-of-the-art LLMs such as GPT-4.

• The paper introduces RAGTruth, a detailed corpus for studying word-level hallucinations in retrieval-augmented language models.
• The research benchmarks various detection methods, showing that fine-tuning smaller models like Llama-2-13B effectively minimizes hallucinations.
• It demonstrates that precise annotations and targeted evaluations can significantly improve trustworthiness in AI-generated outputs.

Analysis of Hallucination in Retrieval-Augmented LLMs with RAGTruth

The paper "RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented LLMs" provides a comprehensive examination of hallucinations in retrieval-augmented generation (RAG) frameworks. RAG is an instrumental technique in minimizing hallucination phenomena inherent in LLMs by integrating retrieval mechanisms into generation processes, thereby enriching the context with relevant information. Despite advancements in RAG, LLMs still occasionally make unsupported claims or contradict retrieved contents, thus necessitating focused research and development in hallucination detection strategies.

Objectives and Contributions

The paper presents RAGTruth, a corpus tailored for studying word-level hallucinations across diverse domains and tasks standard in RAG frameworks. The dataset comprises nearly 18,000 naturally generated responses, meticulously annotated both at individual case and word levels, with emphasis on hallucination intensity. Using RAGTruth, several key contributions are proposed:

1. Dataset Introduction: RAGTruth places a spotlight on word-level hallucination evaluations, distinguishing itself by focusing on naturally generated responses within the RAG context. Compared to previous datasets, RAGTruth represents a significant leap in scale and scope, featuring detailed annotations necessary for in-depth hallucination analysis.
2. Comparative Benchmarks: An extensive comparison of existing hallucination detection methodologies is performed. The analyses focus on both passage-level and word-level indicators, revealing the strengths and limitations of current approaches.
3. Fine-Tuning Efficiency: It demonstrates the potential efficacy of fine-tuning relatively smaller models, such as Llama-2-13B, with RAGTruth, positioning them competitively against prompt-based methods involving state-of-the-art models like GPT-4.
4. Mitigation Success: The paper illustrates the practical advantage of fine-tuned models in reducing hallucinations, benefitting even models known for their low hallucination rates, such as GPT-4.

Hallucination Categories and Analysis

Hallucinations are categorized into evident and subtle conflicts, alongside evident and subtle introductions of baseless information. The paper enumerates how different tasks, like question answering, data-to-text writing, and news summarization, exhibit varying hallucination frequencies. Data-to-text writing, in particular, showed the highest occurrence due to inconsistencies in handling structured data formats.

Discussion of Detection Methods

The experimental setup includes various algorithms such as hallucination detection prompts, SelfCheckGPT, LMvLM, and specialized fine-tuning. Results indicate that fine-tuning using RAGTruth sharply enhances detection prowess, yet challenges still remain at the span-level detection front. Current methods display limitations in precision and recall, highlighting the intricacies of accurately pinpointing hallucinations.

Implications and Future Directions

The paper underscores the persistent challenge of identifying hallucinations within RAG contexts, especially at granular levels. It calls for the continuation of efforts to optimize hallucination detection mechanisms, with RAGTruth heralding future developments in trustworthy LLM applications. Furthermore, the research demonstrates the utility and effectiveness of fine-tuned models, signifying a promising trajectory for model specialization in hallucination detection.

RAGTruth's inception signifies a pivotal shift in corpus creation aimed at tackling hallucination phenomena. By facilitating both empirical and theoretical advances, it serves as a cornerstone for subsequent innovations, methodologically advancing AI towards more reliable and accurate deployment in real-world scenarios.