Large language models have made significant progress in various language tasks, yet they still struggle with complex mathematics. In this paper, we propose ToRA a series of Tool-integrated Reasoning Agents designed to solve challenging mathematical problems by seamlessly integrating natural language reasoning with the utilization of external tools (e.g., computation libraries and symbolic solvers), thereby amalgamating the analytical prowess of language and the computational efficiency of tools. To train ToRA, we curate interactive tool-use trajectories on mathematical datasets, apply imitation learning on the annotations, and propose output space shaping to further refine models' reasoning behavior. As a result, ToRA models significantly outperform open-source models on 10 mathematical reasoning datasets across all scales with 13%-19% absolute improvements on average. Notably, ToRA-7B reaches 44.6% on the competition-level dataset MATH, surpassing the best open-source model WizardMath-70B by 22% absolute. ToRA-Code-34B is also the first open-source model that achieves an accuracy exceeding 50% on MATH, which significantly outperforms GPT-4's CoT result, and is competitive with GPT-4 solving problems with programs. Additionally, we conduct a comprehensive analysis of the benefits and remaining challenges of tool interaction for mathematical reasoning, providing valuable insights for future research.
The paper introduces ToRA, a series of models that integrate natural language reasoning with program-based tool use to improve LLMs' performance in advanced mathematical reasoning tasks.
ToRA combines rationale-based methods and program-based methods using curated tool-use trajectories and imitation learning, resulting in significant improvements on mathematical reasoning datasets.
Experimental results demonstrated that ToRA achieved substantial performance gains, including a 22% improvement over the best previous open-source model on the MATH dataset, and the first open-source model to surpass 50% accuracy on the same dataset.
The paper titled "ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving" addresses the challenges faced by open-source LLMs in advanced mathematical reasoning tasks. The authors introduce ToRA, which stands for Tool-integrated Reasoning Agents, a series of models that integrate natural language reasoning with program-based tool use. This combination aims to leverage the semantic and abstract reasoning capabilities of LLMs alongside the precise computational abilities of external tools.
The authors developed ToRA by enhancing open-source models to interleave natural language reasoning with program-based tool use. This method draws from two primary approaches:
ToRA aims to synergize these methods by generating comprehensive annotations (interactive tool-use trajectories) for mathematical problems and applying imitation learning on these annotations. The key components of the approach include:
ToRA models were evaluated on ten diverse mathematical reasoning datasets. The results indicated significant performance improvements over previous state-of-the-art models. Key findings include:
The results suggest several important implications for AI and mathematical problem solving:
This research paves the way for exploring several future directions in the field of AI and mathematical problem solving:
Overall, ToRA's development and the accompanying results highlight the substantial potential of combining various reasoning strategies to enhance the capabilities of AI models in specialized domains. This research sets a strong foundation for future advancements in AI-driven mathematical reasoning.