TranX + BERT w/mined

Reported on 4 benchmarks across 1 task · 1 paper · 3 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing4 results

Code GenerationonCoNaLa
BLEU· 2022-02-28
34.2
best: 44.32 (PanGu-Coder-FT-I)
SOTA
The impact of lexical and grammatical processing on generating code from natural language arXiv:2202.13972
Code GenerationonCoNaLa
Exact Match Accuracy· 2022-02-28
5.8
best: 13.8 (RoBERTaMarian)
SOTA
The impact of lexical and grammatical processing on generating code from natural language arXiv:2202.13972
Code GenerationonDjango
BLEU Score· 2022-02-28
79.86
best: 90.41 (MarianCG)
SOTA
The impact of lexical and grammatical processing on generating code from natural language arXiv:2202.13972
Code GenerationonDjango
Accuracy· 2022-02-28
81.03
best: 81.83 (MarianCG)
The impact of lexical and grammatical processing on generating code from natural language arXiv:2202.13972