Branch-Train-MiX 4x7B (sampling top-1 expert)

Reported on 1 benchmark across 1 task · 1 paper

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing1 result

Common Sense ReasoningonWinoGrande
Accuracy· 2024-03-12
70.6
best: 96.1 (ST-MoE-32B 269B (fine-tuned))
Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM arXiv:2403.07816