Question Answering on SQuAD2.0 dev

Metric: EM (higher is better)

LeaderboardDataset

Loading chart...

Results

Submit a result

Hide extra data

Sort:

#	Model↕	EM▼	Extra Data	Paper	Date↕	Code
1	XLNet (single model)	87.9	No	XLNet: Generalized Autoregressive Pretraining fo...	2019-06-19	Code
2	XLNet+DSC	87.65	No	Dice Loss for Data-imbalanced NLP Tasks	2019-11-07	Code
3	RoBERTa (no data aug)	86.5	Yes	RoBERTa: A Robustly Optimized BERT Pretraining A...	2019-07-26	Code
4	ALBERT xxlarge	85.1	No	ALBERT: A Lite BERT for Self-supervised Learning...	2019-09-26	Code
5	SG-Net	85.1	No	SG-Net: Syntax-Guided Machine Reading Comprehens...	2019-08-14	Code
6	ALBERT xlarge	83.1	No	ALBERT: A Lite BERT for Self-supervised Learning...	2019-09-26	Code
7	SemBERT large	80.9	No	Semantics-aware BERT for Language Understanding	2019-09-05	Code
8	ALBERT large	79	No	ALBERT: A Lite BERT for Self-supervised Learning...	2019-09-26	Code
9	ALBERT base	76.1	No	ALBERT: A Lite BERT for Self-supervised Learning...	2019-09-26	Code
10	RMR + ELMo (Model-III)	72.3	No	Read + Verify: Machine Reading Comprehension wit...	2018-08-17	-
11	U-Net	70.3	No	U-Net: Machine Reading Comprehension with Unansw...	2018-10-12	Code
12	TinyBERT-6 67M	69.9	No	TinyBERT: Distilling BERT for Natural Language U...	2019-09-23	Code

#1XLNet (single model)SOTA
87.9
EM· 2019-06-19
XLNet: Generalized Autoregressive Pretraining for Language Understanding Code
#2XLNet+DSC
87.65
EM· 2019-11-07
Dice Loss for Data-imbalanced NLP Tasks Code
#3RoBERTa (no data aug)
86.5
EM· Extra Data· 2019-07-26
RoBERTa: A Robustly Optimized BERT Pretraining Approach Code
#4ALBERT xxlarge
85.1
EM· 2019-09-26
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Code
#5SG-Net
85.1
EM· 2019-08-14
SG-Net: Syntax-Guided Machine Reading Comprehension Code
#6ALBERT xlarge
83.1
EM· 2019-09-26
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Code
#7SemBERT large
80.9
EM· 2019-09-05
Semantics-aware BERT for Language Understanding Code
#8ALBERT large
79
EM· 2019-09-26
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Code
#9ALBERT base
76.1
EM· 2019-09-26
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Code
#10RMR + ELMo (Model-III)SOTA
72.3
EM· 2018-08-17
Read + Verify: Machine Reading Comprehension with Unanswerable Questions
#11U-Net
70.3
EM· 2018-10-12
U-Net: Machine Reading Comprehension with Unanswerable Questions Code
#12TinyBERT-6 67M
69.9
EM· 2019-09-23
TinyBERT: Distilling BERT for Natural Language Understanding Code