Metric: Ruby (higher is better)
| # | Model↕ | Ruby▼ | Extra Data | Paper | Date↕ | Code |
|---|---|---|---|---|---|---|
| 1 | cpt-code S | 86.3 | No | Text and Code Embeddings by Contrastive Pre-Trai... | 2022-01-24 | Code |
| 2 | cpt-code M | 85.5 | No | Text and Code Embeddings by Contrastive Pre-Trai... | 2022-01-24 | Code |
| 3 | CodeT5+ 770M | 78 | No | CodeT5+: Open Code Large Language Models for Cod... | 2023-05-13 | Code |
| 4 | CodeT5+ 220M | 77.7 | No | CodeT5+: Open Code Large Language Models for Cod... | 2023-05-13 | Code |
| 5 | GraphCodeBERT | 73.2 | No | GraphCodeBERT: Pre-training Code Representations... | 2020-09-17 | Code |
| 6 | CodeBERT | 70.6 | No | CodeBERT: A Pre-Trained Model for Programming an... | 2020-02-19 | Code |