External Knowledge With API + Reranking

Reported on 2 benchmarks across 1 task · 1 paper · 2 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing2 results

Code GenerationonCoNaLa-Ext
BLEU· 2020-04-20
20.54
best: 35.32 (BART W/ Mined)
SOTA
Incorporating External Knowledge through Pre-training for Natural Language to Code Generation arXiv:2004.09015
Code GenerationonCoNaLa
BLEU· uses extra data· 2020-04-20
32.26
best: 44.32 (PanGu-Coder-FT-I)
SOTA
Incorporating External Knowledge through Pre-training for Natural Language to Code Generation arXiv:2004.09015