Code Generation on CoNaLa

Metric: Exact Match Accuracy (higher is better)

LeaderboardDataset

Loading chart...

Results

Sort:

#	Model↕	Exact Match Accuracy▼	Extra Data	Paper	Date↕	Code
1	RoBERTaMarian	13.8	No	-	-	Code
2	BERTMarian	12.4	No	-	-	Code
3	MarianCG	10.2	No	-	-	Code
4	ELECTRAMarian	10	No	-	-	Code
5	LUKEMarian	7.6	No	-	-	Code
6	TranX + BERT w/mined	5.8	No	The impact of lexical and grammatical processing...	2022-02-28	Code
7	Reranker	2.8	No	-	-	-

#1RoBERTaMarian
13.8
Exact Match Accuracy
No paperCode
#2BERTMarian
12.4
Exact Match Accuracy
No paperCode
#3MarianCG
10.2
Exact Match Accuracy
No paperCode
#4ELECTRAMarian
10
Exact Match Accuracy
No paperCode
#5LUKEMarian
7.6
Exact Match Accuracy
No paperCode
#6TranX + BERT w/minedSOTA
5.8
Exact Match Accuracy· 2022-02-28
The impact of lexical and grammatical processing on generating code from natural language Code
#7Reranker
2.8
Exact Match Accuracy
No paper