SYRELM (GPT-J)

Reported on 4 benchmarks across 4 tasks · 1 paper

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Knowledge Base2 results

Mathematical Question AnsweringonSVAMP
Execution Accuracy· uses extra data· 2023-12-09
40.1
best: 93.9 (GPT-4 (Teaching-Inspired))
Frugal LMs Trained to Invoke Symbolic Solvers Achieve Parameter-Efficient Arithmetic Reasoning arXiv:2312.05571
Mathematical ReasoningonSVAMP
Execution Accuracy· uses extra data· 2023-12-09
40.1
best: 93.9 (GPT-4 (Teaching-Inspired))
Frugal LMs Trained to Invoke Symbolic Solvers Achieve Parameter-Efficient Arithmetic Reasoning arXiv:2312.05571

Natural Language Processing1 result

Question AnsweringonSVAMP
Execution Accuracy· uses extra data· 2023-12-09
40.1
best: 93.9 (GPT-4 (Teaching-Inspired))
Frugal LMs Trained to Invoke Symbolic Solvers Achieve Parameter-Efficient Arithmetic Reasoning arXiv:2312.05571

Reasoning1 result

Math Word Problem SolvingonSVAMP
Execution Accuracy· uses extra data· 2023-12-09
40.1
best: 93.9 (GPT-4 (Teaching-Inspired))
Frugal LMs Trained to Invoke Symbolic Solvers Achieve Parameter-Efficient Arithmetic Reasoning arXiv:2312.05571