Code Generation on CoNaLa
Metric: BLEU (higher is better)
Metric: BLEU (higher is better)
| # | Model↕ | BLEU▼ | Extra Data | Paper | Date↕ | Code |
|---|---|---|---|---|---|---|
| 1 | PanGu-Coder-FT-I | 44.32 | No | - | - | - |
| 2 | RoBERTaMarian | 35.74 | No | - | - | Code |
| 3 | MarianCG | 34.43 | No | - | - | Code |
| 4 | TranX + BERT w/mined | 34.2 | No | The impact of lexical and grammatical processing... | 2022-02-28 | Code |
| 5 | BERT + TAE | 33.41 | Yes | Code Generation from Natural Language with Less ... | 2021-01-01 | Code |
| 6 | BERTMarian | 32.46 | No | - | - | Code |
| 7 | External Knowledge With API + Reranking | 32.26 | Yes | Incorporating External Knowledge through Pre-tra... | 2020-04-20 | Code |
| 8 | External Knowledge With API | 30.69 | Yes | Incorporating External Knowledge through Pre-tra... | 2020-04-20 | Code |
| 9 | BART W/ Mined | 30.55 | No | Reading StackOverflow Encourages Cheating: Addin... | 2021-06-08 | Code |
| 10 | ELECTRAMarian | 30.18 | No | - | - | Code |
| 11 | Reranker | 30.11 | No | - | - | - |
| 12 | LUKEMarian | 29.83 | No | - | - | Code |
| 13 | BART Base | 26.24 | No | Reading StackOverflow Encourages Cheating: Addin... | 2021-06-08 | Code |
| 14 | TranX | 24.3 | No | TRANX: A Transition-based Neural Abstract Syntax... | 2018-10-05 | Code |