Tasks
SotA
Datasets
Papers
Methods
Submit
About
SotA
/
Computer Code
/
Code Completion
/
SAFIM
Code Completion on SAFIM
Metric: Algorithmic (higher is better)
Leaderboard
Dataset
Loading chart...
Results
Submit a result
Export CSV
Sort:
Algorithmic (best first)
Algorithmic (worst first)
Date (newest first)
Date (oldest first)
Model name (A→Z)
#
Model
↕
Algorithmic
▼
Extra Data
Paper
Date
↕
Code
1
deepseek-coder-33b-base
60.78
No
Evaluation of LLMs on Syntax-Aware Code Fill-in-...
2024-03-07
Code
2
deepseek-coder-6.7b-base
54.74
No
Evaluation of LLMs on Syntax-Aware Code Fill-in-...
2024-03-07
Code
3
starcoderbase
44.11
No
Evaluation of LLMs on Syntax-Aware Code Fill-in-...
2024-03-07
Code
4
gpt-4-1106-preview
42.11
No
Evaluation of LLMs on Syntax-Aware Code Fill-in-...
2024-03-07
Code
5
CodeLlama-13b-hf
41.41
No
Evaluation of LLMs on Syntax-Aware Code Fill-in-...
2024-03-07
Code
6
deepseek-coder-1.3b-base
41.2
No
Evaluation of LLMs on Syntax-Aware Code Fill-in-...
2024-03-07
Code
7
CodeLlama-34b-hf
38.55
No
Evaluation of LLMs on Syntax-Aware Code Fill-in-...
2024-03-07
Code
8
CodeLlama-7b-hf
34.68
No
Evaluation of LLMs on Syntax-Aware Code Fill-in-...
2024-03-07
Code
9
gpt-3.5-turbo-0301
31.24
No
Evaluation of LLMs on Syntax-Aware Code Fill-in-...
2024-03-07
Code
10
codegen-16B-multi
25.94
No
Evaluation of LLMs on Syntax-Aware Code Fill-in-...
2024-03-07
Code
11
incoder-6B
25.16
No
Evaluation of LLMs on Syntax-Aware Code Fill-in-...
2024-03-07
Code
12
codegen-6B-multi
23.6
No
Evaluation of LLMs on Syntax-Aware Code Fill-in-...
2024-03-07
Code
13
codegen-2B-multi
23.49
No
Evaluation of LLMs on Syntax-Aware Code Fill-in-...
2024-03-07
Code
14
incoder-1B
21.06
No
Evaluation of LLMs on Syntax-Aware Code Fill-in-...
2024-03-07
Code
15
codegen-350M-multi
16.3
No
Evaluation of LLMs on Syntax-Aware Code Fill-in-...
2024-03-07
Code
#1
deepseek-coder-33b-base
SOTA
60.78
Algorithmic
· 2024-03-07
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks
Code
#2
deepseek-coder-6.7b-base
54.74
Algorithmic
· 2024-03-07
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks
Code
#3
starcoderbase
44.11
Algorithmic
· 2024-03-07
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks
Code
#4
gpt-4-1106-preview
42.11
Algorithmic
· 2024-03-07
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks
Code
#5
CodeLlama-13b-hf
41.41
Algorithmic
· 2024-03-07
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks
Code
#6
deepseek-coder-1.3b-base
41.2
Algorithmic
· 2024-03-07
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks
Code
#7
CodeLlama-34b-hf
38.55
Algorithmic
· 2024-03-07
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks
Code
#8
CodeLlama-7b-hf
34.68
Algorithmic
· 2024-03-07
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks
Code
#9
gpt-3.5-turbo-0301
31.24
Algorithmic
· 2024-03-07
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks
Code
#10
codegen-16B-multi
25.94
Algorithmic
· 2024-03-07
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks
Code
#11
incoder-6B
25.16
Algorithmic
· 2024-03-07
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks
Code
#12
codegen-6B-multi
23.6
Algorithmic
· 2024-03-07
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks
Code
#13
codegen-2B-multi
23.49
Algorithmic
· 2024-03-07
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks
Code
#14
incoder-1B
21.06
Algorithmic
· 2024-03-07
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks
Code
#15
codegen-350M-multi
16.3
Algorithmic
· 2024-03-07
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks
Code