MapCoder (GPT-4)

Reported on 4 benchmarks across 1 task · 1 paper · 3 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing4 results

Code GenerationonCodeContests
Test Set pass@1· 2024-05-18
28.5
best: 58.18 (EG-CFG (DeepSeek-V3-0324))
SOTA
MapCoder: Multi-Agent Code Generation for Competitive Problem Solving arXiv:2405.11403
Code GenerationonCodeContests
Test Set pass@5· 2024-05-18
35.2
SOTA
MapCoder: Multi-Agent Code Generation for Competitive Problem Solving arXiv:2405.11403
Code GenerationonCodeContests
Val Set pass@1· 2024-05-18
28.5
SOTA
MapCoder: Multi-Agent Code Generation for Competitive Problem Solving arXiv:2405.11403
Code GenerationonMBPP
Accuracy· 2024-05-18
83.1
best: 96.6 (EG-CFG (DeepSeek-V3-0324))
MapCoder: Multi-Agent Code Generation for Competitive Problem Solving arXiv:2405.11403