A Closer Look at the Self-Verification Abilities of Large Language Models in Logical Reasoning

Abstract: Logical reasoning has been an ongoing pursuit in the field of AI. Despite significant advancements made by LLMs, they still struggle with complex logical reasoning problems. To enhance reasoning performance, one promising direction is scalable oversight, which requires LLMs to identify their own errors and then improve by themselves. Various self-verification methods have been proposed in pursuit of this goal. Nevertheless, whether existing models understand their own errors well is still under investigation. In this paper, we take a closer look at the self-verification abilities of LLMs in the context of logical reasoning, focusing on their ability to identify logical fallacies accurately. We introduce a dataset, FALLACIES, containing 232 types of reasoning fallacies categorized in a hierarchical taxonomy. By conducting exhaustive experiments on FALLACIES, we obtain comprehensive and detailed analyses of a series of models on their verification abilities. Our main findings suggest that existing LLMs could struggle to identify fallacious reasoning steps accurately and may fall short of guaranteeing the validity of self-verification methods. Drawing from these observations, we offer suggestions for future research and practical applications of self-verification methods.

• The paper demonstrates that LLMs exhibit self-verification capabilities but often miss logical fallacies.
• It introduces a comprehensive fallacies dataset with 4,640 reasoning steps across 232 types of fallacies.
• Experimental results reveal that even state-of-the-art models achieve less than 80% accuracy in detecting fallacies.

A Closer Look at the Self-Verification Abilities of LLMs in Logical Reasoning

Introduction

The paper "A Closer Look at the Self-Verification Abilities of LLMs in Logical Reasoning" addresses the critical issue of logical reasoning within the framework of LLMs. The paper identifies a prevalent problem where LLMs, despite their emergent reasoning abilities, frequently generate invalid reasoning steps that fall into logical fallacies. To tackle this, a scalable oversight approach known as self-verification is explored, positing that LLMs can self-evaluate and correct erroneous logic.

Figure 1: The self-verification approach requires LLMs to identify the fallacious steps in their own reasoning process. However, LLMs might be susceptible to certain types of fallacies and fail to identify them, leading to the potential failure of self-verification.

Fallacies Dataset Introduction

The authors introduce a comprehensive dataset named "Fallacies," composed of 4,640 logical reasoning steps encompassing 232 types of fallacies categorized within a hierarchical taxonomy. The aim is to evaluate the self-verification abilities of LLMs by focusing on their performance in identifying these fallacies.

Figure 2: The hierarchical taxonomy of fallacies. For each sub-category, we present its definition and an example of a fallacy within the sub-category. We use square brackets to indicate the premises and conclusions.

Methodology and Experiments

The dataset allows for an in-depth analysis of various LLMs such as GPT-4, GPT-3.5, and others. Models were evaluated based on their ability to identify both the presence of logical fallacies and the specific types when present. Results showed that even state-of-the-art LLMs struggled, with most achieving an accuracy of less than 80% in identifying fallacious steps, indicating significant room for improvement.

A notable finding is the performance disparity among LLMs in identifying different categories of fallacies. Formal fallacies, which focus more on the logical structure, posed more of a challenge than informal fallacies that dealt with content. This indicates that logical structure understanding remains a bottleneck in logical reasoning capabilities of LLMs.

Figure 3: The definition of the fallacy of "Affirming the Consequent", one of 232 types of fallacies in our dataset.

Implications for LLM Development

The study underscores the limitations of relying solely on self-verification methods for logical reasoning within LLMs. It suggests the need for more tailored approaches that enhance a model's understanding of logical structures and fallacies. Additionally, it highlights the importance of comprehensive datasets that include a wide range of reasoning errors to facilitate robust model training and evaluation.

Given the current limitations observed, future work may explore methodologies like multi-agent debate systems or human-in-the-loop interactions to augment self-verification strategies, possibly bridging the gap between human and machine logical reasoning capabilities.

Conclusion

In conclusion, while self-verification presents a promising avenue for improving the reasoning capabilities of LLMs, the current limitations necessitate a cautious approach to its application. The research provides valuable insights and a robust dataset for further work in enhancing logical reasoning in AI, emphasizing the need for ongoing refinement in both theoretical and practical domains. This paper serves as a reminder of the complexities inherent in mimicking human reasoning and the continuous effort required to achieve this goal in artificial systems.