Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Minlie Huang, Nan Duan, Weizhu Chen
Large language models have made significant progress in various language tasks, yet they still struggle with complex mathematics. In this paper, we propose ToRA a series of Tool-integrated Reasoning Agents designed to solve challenging mathematical problems by seamlessly integrating natural language reasoning with the utilization of external tools (e.g., computation libraries and symbolic solvers), thereby amalgamating the analytical prowess of language and the computational efficiency of tools. To train ToRA, we curate interactive tool-use trajectories on mathematical datasets, apply imitation learning on the annotations, and propose output space shaping to further refine models' reasoning behavior. As a result, ToRA models significantly outperform open-source models on 10 mathematical reasoning datasets across all scales with 13%-19% absolute improvements on average. Notably, ToRA-7B reaches 44.6% on the competition-level dataset MATH, surpassing the best open-source model WizardMath-70B by 22% absolute. ToRA-Code-34B is also the first open-source model that achieves an accuracy exceeding 50% on MATH, which significantly outperforms GPT-4's CoT result, and is competitive with GPT-4 solving problems with programs. Additionally, we conduct a comprehensive analysis of the benefits and remaining challenges of tool interaction for mathematical reasoning, providing valuable insights for future research.
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Question Answering | MATH | Accuracy | 60 | ToRA-Code 34B model (w/ code, SC, k=50) |
| Question Answering | MATH | Parameters (Billions) | 34 | ToRA-Code 34B model (w/ code, SC, k=50) |
| Question Answering | MATH | Accuracy | 56.9 | ToRA 70B (w/ code, SC, k=50) |
| Question Answering | MATH | Parameters (Billions) | 70 | ToRA 70B (w/ code, SC, k=50) |
| Question Answering | MATH | Accuracy | 50.8 | ToRA-Code 34B (w/ code) |
| Question Answering | MATH | Parameters (Billions) | 34 | ToRA-Code 34B (w/ code) |
| Question Answering | MATH | Accuracy | 49.7 | ToRA 70B (w/ code) |
| Question Answering | MATH | Parameters (Billions) | 70 | ToRA 70B (w/ code) |
| Question Answering | MATH | Accuracy | 48.1 | ToRA-Code 13B (w/ code) |
| Question Answering | MATH | Parameters (Billions) | 13 | ToRA-Code 13B (w/ code) |
| Question Answering | MATH | Accuracy | 44.6 | ToRA-Code 7B (w/ code) |
| Question Answering | MATH | Parameters (Billions) | 7 | ToRA-Code 7B (w/ code) |
| Question Answering | MATH | Accuracy | 43 | ToRA 13B (w/ code) |
| Question Answering | MATH | Parameters (Billions) | 13 | ToRA 13B (w/ code) |
| Question Answering | MATH | Accuracy | 40.1 | ToRA 7B (w/ code) |
| Question Answering | MATH | Parameters (Billions) | 7 | ToRA 7B (w/ code) |
| Math Word Problem Solving | MATH | Accuracy | 60 | ToRA-Code 34B model (w/ code, SC, k=50) |
| Math Word Problem Solving | MATH | Parameters (Billions) | 34 | ToRA-Code 34B model (w/ code, SC, k=50) |
| Math Word Problem Solving | MATH | Accuracy | 56.9 | ToRA 70B (w/ code, SC, k=50) |
| Math Word Problem Solving | MATH | Parameters (Billions) | 70 | ToRA 70B (w/ code, SC, k=50) |
| Math Word Problem Solving | MATH | Accuracy | 50.8 | ToRA-Code 34B (w/ code) |
| Math Word Problem Solving | MATH | Parameters (Billions) | 34 | ToRA-Code 34B (w/ code) |
| Math Word Problem Solving | MATH | Accuracy | 49.7 | ToRA 70B (w/ code) |
| Math Word Problem Solving | MATH | Parameters (Billions) | 70 | ToRA 70B (w/ code) |
| Math Word Problem Solving | MATH | Accuracy | 48.1 | ToRA-Code 13B (w/ code) |
| Math Word Problem Solving | MATH | Parameters (Billions) | 13 | ToRA-Code 13B (w/ code) |
| Math Word Problem Solving | MATH | Accuracy | 44.6 | ToRA-Code 7B (w/ code) |
| Math Word Problem Solving | MATH | Parameters (Billions) | 7 | ToRA-Code 7B (w/ code) |
| Math Word Problem Solving | MATH | Accuracy | 43 | ToRA 13B (w/ code) |
| Math Word Problem Solving | MATH | Parameters (Billions) | 13 | ToRA 13B (w/ code) |
| Math Word Problem Solving | MATH | Accuracy | 40.1 | ToRA 7B (w/ code) |
| Math Word Problem Solving | MATH | Parameters (Billions) | 7 | ToRA 7B (w/ code) |
| Mathematical Question Answering | MATH | Accuracy | 60 | ToRA-Code 34B model (w/ code, SC, k=50) |
| Mathematical Question Answering | MATH | Parameters (Billions) | 34 | ToRA-Code 34B model (w/ code, SC, k=50) |
| Mathematical Question Answering | MATH | Accuracy | 56.9 | ToRA 70B (w/ code, SC, k=50) |
| Mathematical Question Answering | MATH | Parameters (Billions) | 70 | ToRA 70B (w/ code, SC, k=50) |
| Mathematical Question Answering | MATH | Accuracy | 50.8 | ToRA-Code 34B (w/ code) |
| Mathematical Question Answering | MATH | Parameters (Billions) | 34 | ToRA-Code 34B (w/ code) |
| Mathematical Question Answering | MATH | Accuracy | 49.7 | ToRA 70B (w/ code) |
| Mathematical Question Answering | MATH | Parameters (Billions) | 70 | ToRA 70B (w/ code) |
| Mathematical Question Answering | MATH | Accuracy | 48.1 | ToRA-Code 13B (w/ code) |
| Mathematical Question Answering | MATH | Parameters (Billions) | 13 | ToRA-Code 13B (w/ code) |
| Mathematical Question Answering | MATH | Accuracy | 44.6 | ToRA-Code 7B (w/ code) |
| Mathematical Question Answering | MATH | Parameters (Billions) | 7 | ToRA-Code 7B (w/ code) |
| Mathematical Question Answering | MATH | Accuracy | 43 | ToRA 13B (w/ code) |
| Mathematical Question Answering | MATH | Parameters (Billions) | 13 | ToRA 13B (w/ code) |
| Mathematical Question Answering | MATH | Accuracy | 40.1 | ToRA 7B (w/ code) |
| Mathematical Question Answering | MATH | Parameters (Billions) | 7 | ToRA 7B (w/ code) |
| Mathematical Reasoning | MATH | Accuracy | 60 | ToRA-Code 34B model (w/ code, SC, k=50) |
| Mathematical Reasoning | MATH | Parameters (Billions) | 34 | ToRA-Code 34B model (w/ code, SC, k=50) |
| Mathematical Reasoning | MATH | Accuracy | 56.9 | ToRA 70B (w/ code, SC, k=50) |
| Mathematical Reasoning | MATH | Parameters (Billions) | 70 | ToRA 70B (w/ code, SC, k=50) |
| Mathematical Reasoning | MATH | Accuracy | 50.8 | ToRA-Code 34B (w/ code) |
| Mathematical Reasoning | MATH | Parameters (Billions) | 34 | ToRA-Code 34B (w/ code) |
| Mathematical Reasoning | MATH | Accuracy | 49.7 | ToRA 70B (w/ code) |
| Mathematical Reasoning | MATH | Parameters (Billions) | 70 | ToRA 70B (w/ code) |
| Mathematical Reasoning | MATH | Accuracy | 48.1 | ToRA-Code 13B (w/ code) |
| Mathematical Reasoning | MATH | Parameters (Billions) | 13 | ToRA-Code 13B (w/ code) |
| Mathematical Reasoning | MATH | Accuracy | 44.6 | ToRA-Code 7B (w/ code) |
| Mathematical Reasoning | MATH | Parameters (Billions) | 7 | ToRA-Code 7B (w/ code) |
| Mathematical Reasoning | MATH | Accuracy | 43 | ToRA 13B (w/ code) |
| Mathematical Reasoning | MATH | Parameters (Billions) | 13 | ToRA 13B (w/ code) |
| Mathematical Reasoning | MATH | Accuracy | 40.1 | ToRA 7B (w/ code) |
| Mathematical Reasoning | MATH | Parameters (Billions) | 7 | ToRA 7B (w/ code) |
| Arithmetic Reasoning | GSM8K | Accuracy | 88.3 | ToRA-70B (SC, k=50) |
| Arithmetic Reasoning | GSM8K | Parameters (Billion) | 70 | ToRA-70B (SC, k=50) |
| Arithmetic Reasoning | GSM8K | Accuracy | 85.1 | ToRA-Code-34B (SC, k=50) |
| Arithmetic Reasoning | GSM8K | Parameters (Billion) | 34 | ToRA-Code-34B (SC, k=50) |
| Arithmetic Reasoning | GSM8K | Accuracy | 84.3 | ToRA 70B |
| Arithmetic Reasoning | GSM8K | Parameters (Billion) | 70 | ToRA 70B |
| Arithmetic Reasoning | GSM8K | Accuracy | 80.7 | ToRA-Code 34B |
| Arithmetic Reasoning | GSM8K | Parameters (Billion) | 34 | ToRA-Code 34B |
| Arithmetic Reasoning | GSM8K | Accuracy | 75.8 | ToRA-Code 13B |
| Arithmetic Reasoning | GSM8K | Parameters (Billion) | 13 | ToRA-Code 13B |
| Arithmetic Reasoning | GSM8K | Accuracy | 72.6 | ToRA-Code 7B |
| Arithmetic Reasoning | GSM8K | Parameters (Billion) | 7 | ToRA-Code 7B |