Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languages (2310.14799v1)

Abstract: Chain-of-thought (CoT) is capable of eliciting models to explicitly generate reasoning paths, thus promoting reasoning accuracy and attracting increasing attention. Specifically, zero-shot CoT achieves remarkable improvements in a wide range of reasoning tasks by simply instructing the LLM with the prompt "Let's think step by step!". Despite the success of zero-shot CoT, the existing zero-shot prompting techniques remain limited to a single language, making it challenging to generalize to other languages and hindering global development. In this work, we introduce cross-lingual prompting (CLP), aiming to improve zero-shot CoT reasoning across languages. Specifically, CLP consists of two main components: (1) cross-lingual alignment prompting and (2) task-specific solver prompting. The cross-lingual alignment prompting is responsible for aligning representations across different languages, whereas the task-specific solver prompting is used to generate the final chain of thoughts and results for the reasoning task. In addition, we further introduce cross-lingual self-consistent prompting (CLSP) to ensemble different reasoning paths across languages. Our experimental evaluations on several benchmarks demonstrate that CLP and CLSP significantly outperform the existing prompting methods and achieve state-of-the-art performance. We hope this work will inspire further breakthroughs in cross-lingual CoT.

• The paper introduces cross-lingual prompting (CLP) and cross-lingual self-consistent prompting (CLSP) to enhance zero-shot chain-of-thought reasoning across languages.
• It employs a two-stage strategy that aligns language representations and integrates diverse reasoning paths, achieving up to a 6.1% performance boost.
• The methods improve the fidelity and robustness of LLM reasoning, paving the way for more adaptable and multilingual AI systems.

Cross-lingual Prompting: Enhancing Zero-shot Chain-of-Thought Reasoning

The paper discusses a novel technique termed Cross-lingual Prompting (CLP) and its extension, Cross-lingual Self-consistent Prompting (CLSP), aimed at improving zero-shot Chain-of-Thought (CoT) reasoning across multiple languages. The motivation arises from the limitation of existing zero-shot CoT techniques predominantly focusing on a single language, thereby hindering their applicability in a global context.

Introduction to Cross-lingual Prompting

CLP is designed to bridge the gap in reasoning capabilities across different languages by employing two core components: cross-lingual alignment prompting and task-specific solver prompting. The cross-lingual alignment prompting component aims to harmonize representations across languages, while the task-specific solver prompting leverages these aligned representations to construct the CoT necessary for task-solving.

Figure 1: Traditional Chain-of-Though (CoT) vs. Cross-lingual CoT.

In tandem with CLP, the paper introduces CLSP, which integrates reasoning paths from multiple languages, enabling a more robust ensemble of diverse language-specific reasoning insights. The experimental evaluations reveal that these strategies significantly outperform existing prompting methods, achieving state-of-the-art performance improvements.

Framework and Implementation

Cross-lingual Alignment Prompting

The CLP framework involves initiating the LLM with a cross-lingual alignment task. This is accomplished by formulating requests in a source language and task-specific prompts in English, allowing the model to align language representations effectively before solving the reasoning task.

Figure 2: The main framework of CLP (a) and CLSP (b). CLP involves cross-lingual alignment prompting and task-specific solver prompting; CLSP integrates multiple reasoning paths.

Task-specific Solver Prompting

Once alignment is achieved, CLP applies task-specific solver prompting. This component facilitates multi-step reasoning by engaging the model as an expert in the target language, resolving the aligned task efficiently.

Cross-lingual Self-consistent Prompting

CLSP further enhances performance by leveraging a voting mechanism to integrate various reasoning paths and selecting the most consistent outcomes across languages. This method significantly boosts accuracy, demonstrating the benefits of combining knowledge across linguistic domains.

Results and Analysis

The experiments on several benchmarks, including MGSM, highlight that CLP achieves state-of-the-art accuracy gains, exceeding baseline models by over 1.8%. CLSP further amplifies these results through its unique ensemble methodology, delivering an average improvement of 6.1% compared to CLP alone.

Figure 3: The accuracy comparison between two-stage interactive prompting and single-turn prompting.

Analysis of reasoning paths using the ROSCOE framework showed that CLP enhances the faithfulness and informativeness of CoT outputs. Two-stage interactive prompting demonstrated substantial improvements over single-turn prompting procedures, reinforcing the importance of structured, step-by-step engagement with LLMs.

Robustness and Generality

The robustness of CLP is evident across various prompts and models, maintaining consistent performance improvements. Furthermore, its application extends to other benchmarks, showcasing its adaptability to diverse languages and reasoning tasks.

Figure 4: The Acc. comparison on other benchmarks.

Further exploration into the use of in-context learning (ICL) alongside CLP reveals synergistic outcomes, where alignment prompts and solver prompts enhance performance when combined with ICL methodologies.

Conclusion

This research establishes a profound advancement in cross-lingual CoT reasoning through CLP and CLSP frameworks. By addressing cross-lingual transferability challenges, the proposed methods significantly enrich reasoning capabilities across languages without additional language-specific training data. These findings open avenues for even broader application of LLMs in multilingual contexts, propelling future research in the domain of cross-lingual AI systems.

In conclusion, CLP and CLSP not only improve cross-lingual reasoning accuracy but also demonstrate the potential for robust, language-agnostic AI functionalities in complex reasoning tasks. The work provides a solid foundation for future explorations into enhancing multilingual capabilities of LLMs.