Orca 2-13B

Reported on 6 benchmarks across 5 tasks · 1 paper · 2 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing4 results

Question AnsweringonAGI Eval
Accuracy· 2023-11-18
49.93
SOTA
Orca 2: Teaching Small Language Models How to Reason arXiv:2311.11045
Common Sense ReasoningonBIG-bench
Accuracy· 2023-11-18
86.86
SOTA
Orca 2: Teaching Small Language Models How to Reason arXiv:2311.11045
Reading ComprehensiononRACE
Accuracy· 2023-11-18
82.87
best: 91.4 (ALBERT (Ensemble))
Orca 2: Teaching Small Language Models How to Reason arXiv:2311.11045
Question AnsweringonDROP Test
F1· 2023-11-18
57.97
best: 88.38 (QDGAT (ensemble))
Orca 2: Teaching Small Language Models How to Reason arXiv:2311.11045

Methodology2 results

Transfer LearningonBBH-nlp
Average (%)· 2023-11-18
50.18
best: 86.3 (Qwen2.5-72B)
Orca 2: Teaching Small Language Models How to Reason arXiv:2311.11045
Multi-Task LearningonBBH-nlp
Average (%)· 2023-11-18
50.18
best: 86.3 (Qwen2.5-72B)
Orca 2: Teaching Small Language Models How to Reason arXiv:2311.11045