Tasks SotA Datasets Papers Methods Submit About

Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable Benchmarks All SotA Datasets Papers Methods

Community

Submit Results About

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/Branch-Train-MiX 4x7B (sampling top-2 experts)

Branch-Train-MiX 4x7B (sampling top-2 experts)

Reported on 7 benchmarks across 6 tasks · 1 paper

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing3 results

Question AnsweringonTriviaQA
EM· 2024-03-12
57.1
best: 87.5 (Claude 2 (few-shot, k=5))
Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM arXiv:2403.07816
Question AnsweringonMATH
Accuracy· 2024-03-12
17.8
best: 89.7 (Gemini 2.0 Flash Experimental)
Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM arXiv:2403.07816
Code GenerationonMBPP
Accuracy· 2024-03-12
39.4
best: 96.6 (EG-CFG (DeepSeek-V3-0324))
Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM arXiv:2403.07816

Reasoning2 results

Math Word Problem SolvingonMATH
Accuracy· 2024-03-12
17.8
best: 89.7 (Gemini 2.0 Flash Experimental)
Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM arXiv:2403.07816
Arithmetic ReasoningonGSM8K
Accuracy· 2024-03-12
37.1
best: 97.72 (Claude 3.5 Sonnet (HPT))
Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM arXiv:2403.07816

Knowledge Base2 results

Mathematical Question AnsweringonMATH
Accuracy· 2024-03-12
17.8
best: 89.7 (Gemini 2.0 Flash Experimental)
Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM arXiv:2403.07816
Mathematical ReasoningonMATH
Accuracy· 2024-03-12
17.8
best: 89.7 (Gemini 2.0 Flash Experimental)
Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM arXiv:2403.07816