Code Generation on APPS
Metric: Introductory Pass@1 (higher is better)
Metric: Introductory Pass@1 (higher is better)
| # | Model↕ | Introductory Pass@1▼ | Extra Data | Paper | Date↕ | Code |
|---|---|---|---|---|---|---|
| 1 | LPW (GPT-4o) | 87.2 | No | Planning-Driven Programming: A Large Language Mo... | 2024-11-21 | Code |
| 2 | MoTCoder-32B-V1.5 | 68.44 | No | MoTCoder: Elevating Large Language Models with M... | 2023-12-26 | Code |
| 3 | MoTCoder-7B-V1.5 | 54.26 | No | MoTCoder: Elevating Large Language Models with M... | 2023-12-26 | Code |
| 4 | deepseek-ai/deepseek-coder-6.7b-instruct | 33.8 | No | DeepSeek-Coder: When the Large Language Model Me... | 2024-01-25 | Code |
| 5 | code-davinci-002 175B | 31.92 | No | CodeT: Code Generation with Generated Tests | 2022-07-21 | Code |
| 6 | WizardCoder-15b | 26.29 | No | CodeChain: Towards Modular Code Generation Throu... | 2023-10-13 | Code |
| 7 | CodeSim (GPT4) | 26.04 | No | CODESIM: Multi-Agent Code Generation and Problem... | 2025-02-08 | Code |
| 8 | CodeRL+CodeT5 | 20 | No | CodeRL: Mastering Code Generation through Pretra... | 2022-07-05 | Code |