General Purpose Verification for Chain of Thought Prompting (2405.00204v1)

Abstract: Many of the recent capabilities demonstrated by LLMs arise primarily from their ability to exploit contextual information. In this paper, we explore ways to improve reasoning capabilities of LLMs through (1) exploration of different chains of thought and (2) validation of the individual steps of the reasoning process. We propose three general principles that a model should adhere to while reasoning: (i) Relevance, (ii) Mathematical Accuracy, and (iii) Logical Consistency. We apply these constraints to the reasoning steps generated by the LLM to improve the accuracy of the final generation. The constraints are applied in the form of verifiers: the model itself is asked to verify if the generated steps satisfy each constraint. To further steer the generations towards high-quality solutions, we use the perplexity of the reasoning steps as an additional verifier. We evaluate our method on 4 distinct types of reasoning tasks, spanning a total of 9 different datasets. Experiments show that our method is always better than vanilla generation, and, in 6 out of the 9 datasets, it is better than best-of N sampling which samples N reasoning chains and picks the lowest perplexity generation.

• The paper introduces a verification framework that enforces relevance, mathematical accuracy, and logical consistency in chain-of-thought prompting.
• It employs dedicated verifiers and a geometric mean aggregation strategy to systematically score reasoning steps and improve performance on benchmarks like GSM8k.
• Human evaluations correlate with verifier metrics, highlighting both the framework’s benefits and potential areas for further refinement in LLM reasoning.

General Purpose Verification for Chain of Thought Prompting

Introduction

The paper "General Purpose Verification for Chain of Thought Prompting" describes a methodology aimed at enhancing the reasoning capabilities of LLMs. The authors focus on two main areas: exploring diverse chains of thought and validating individual reasoning steps. They propose three key principles for ensuring sound reasoning: relevance, mathematical accuracy, and logical consistency. These principles are enforced through dedicated verifiers, which check if each reasoning step adheres to these constraints. Additionally, the perplexity of the reasoning steps is used as an auxiliary verifier to guide the models towards producing high-quality solutions.

Methodology

Solution Generation

The approach utilizes a solution generator—typically an LLM—that operates on a prompt to generate a sequence of reasoning steps. The generation process is oriented around the chain-of-thought prompting method, where intermediate reasoning steps are generated, potentially allowing more robust solutions as opposed to a single-step generation process. The prompt encourages the LLM to operate in a manner akin to step-by-step human reasoning.

Step Verification

Verification of reasoning steps operates on three principles:

1. Relevance: Ensures that each step contributes meaningfully to the final solution, avoiding irrelevant or unrelated information.
2. Mathematical Accuracy: Verifies the correctness of any mathematical calculations in a reasoning step, employing structured extraction of mathematical expressions to validate them systematically.
3. Logical Consistency: Assesses whether a reasoning step contradicts prior steps, ensuring coherence throughout the reasoning chain.

   Figure 1: An example of each of our proposed verifiers applied to a given question and previous steps.

Aggregation Strategy

To derive a comprehensive score for a complete reasoning chain, the verifiers' outputs are aggregated. Each reasoning step is scored individually, and a geometric mean is used to amalgamate these step-level scores into an overall chain score. This approach ensures that individual inaccuracies are identified and mitigated across the reasoning process.

Experimental Results

Datasets and Results

The evaluation spans various reasoning tasks across nine datasets, including GSM8k and BigBench Date Understanding. Results demonstrate that using the proposed verification framework yields better performance than traditional generation methods, with significant improvements highlighted over baseline models, as shown in self-consistency experiments.

Figure 2: Correlation between the scores of our proposed verifiers and the assessment of human annotators over the three reasoning principles explored in this work. All correlations have a p-value of less than 0.0001.

Human Evaluation

To assess the alignment of the verification framework with human judgment, extensive human evaluation was conducted. Correlations between the verifiers' scores and human assessments indicated that while the principles capture a substantial portion of the reasoning quality, individual verifiers still have room for enhancement, indicated by moderate positive correlations.

Implications and Future Work

The authors conclude that though current verification methodologies provide notable improvements in reasoning tasks, there is potential for further refinement in verifier accuracy and applicability. Developing more precise and adapting these verifiers for specific domain tasks could offer substantial benefits. Additionally, investigating alternative strategies for constructing verifiers that can maintain performance while being computationally efficient remains a vital avenue for future exploration.

Conclusion

The verification framework for chain-of-thought prompting provides a novel approach to refine LLM reasoning by integrating model-agnostic and task-agnostic principles that align with human evaluation criteria. By demonstrating measurable performance gains across various datasets, this paper establishes a foundation for more robust and interpretable LLM applications in complex reasoning tasks.

This paper opens pathways to leveraging structured verification methods, potentially translating into better decision-making and inference capabilities in LLMs across diverse applied contexts.