Jiutian-大模型

Reported on 5 benchmarks across 4 tasks

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Methodology2 results

Transfer LearningonBBH-nlp
Average (%)
86.1
best: 86.3 (Qwen2.5-72B)
Multi-Task LearningonBBH-nlp
Average (%)
86.1
best: 86.3 (Qwen2.5-72B)

Reasoning2 results

Arithmetic ReasoningonGSM8K
Accuracy
95.2
best: 97.72 (Claude 3.5 Sonnet (HPT))
Arithmetic ReasoningonGSM8K
Parameters (Billion)
75
best: 540 (PaLM 540B (Self Improvement, Self Consistency))

Natural Language Processing1 result

Code GenerationonMBPP
Accuracy
90
best: 96.6 (EG-CFG (DeepSeek-V3-0324))