TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

SotA/Natural Language Processing/Code Generation/RES-Q

Code Generation on RES-Q

Metric: pass@1 (higher is better)

LeaderboardDataset
Loading chart...

Results

Submit a result
#Model↕pass@1▼Extra DataPaperDate↕Code
1QurrentOS-coder + Claude 3.5 Sonnet58NoRES-Q: Evaluating Code-Editing Large Language Mo...2024-06-24Code
2QurrentOS-coder + GPT-4o46NoRES-Q: Evaluating Code-Editing Large Language Mo...2024-06-24Code
3QurrentOS-coder + GPT-4 Turbo37NoRES-Q: Evaluating Code-Editing Large Language Mo...2024-06-24Code
4QurrentOS-coder + Claude 3 Opus36NoRES-Q: Evaluating Code-Editing Large Language Mo...2024-06-24Code
5QurrentOS-coder + GPT-430NoRES-Q: Evaluating Code-Editing Large Language Mo...2024-06-24Code
6QurrentOS-coder + Gemini 1.5 Pro30NoRES-Q: Evaluating Code-Editing Large Language Mo...2024-06-24Code
7QurrentOS-coder + DeepSeek-Coder-V229NoRES-Q: Evaluating Code-Editing Large Language Mo...2024-06-24Code
8QurrentOS-coder + Llama 3 70b20NoRES-Q: Evaluating Code-Editing Large Language Mo...2024-06-24Code
9QurrentOS-coder + Qwen-72B-Instruct18NoRES-Q: Evaluating Code-Editing Large Language Mo...2024-06-24Code