Metric: Competition Pass@1 (higher is better)
| # | Model↕ | Competition Pass@1▼ | Extra Data | Paper | Date↕ | Code |
|---|---|---|---|---|---|---|
| 1 | LPW (GPT-4o) | 34.8 | No | Planning-Driven Programming: A Large Language Mo... | 2024-11-21 | Code |
| 2 | CodeRL+CodeT5 | 33.3 | No | CodeRL: Mastering Code Generation through Pretra... | 2022-07-05 | Code |
| 3 | MoTCoder-32B-V1.5 | 27.84 | No | MoTCoder: Elevating Large Language Models with M... | 2023-12-26 | Code |
| 4 | MoTCoder-7B-V1.5 | 21.18 | No | MoTCoder: Elevating Large Language Models with M... | 2023-12-26 | Code |
| 5 | deepseek-ai/deepseek-coder-6.7b-instruct | 11.09 | No | DeepSeek-Coder: When the Large Language Model Me... | 2024-01-25 | Code |
| 6 | WizardCoder-15b | 3.75 | No | CodeChain: Towards Modular Code Generation Throu... | 2023-10-13 | Code |
| 7 | CodeSim (GPT4) | 0.81 | No | CODESIM: Multi-Agent Code Generation and Problem... | 2025-02-08 | Code |