TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

SotA/Natural Language Processing/Question Answering/SQuAD1.1 dev

Question Answering on SQuAD1.1 dev

Metric: EM (higher is better)

LeaderboardDataset
Loading chart...

Results

Submit a result
#Model↕EM▼Extra DataPaperDate↕Code
1T5-11B90.06YesExploring the Limits of Transfer Learning with a...2019-10-23Code
2LUKE89.8YesLUKE: Deep Contextualized Entity Representations...2020-10-02Code
3XLNet+DSC89.79YesDice Loss for Data-imbalanced NLP Tasks2019-11-07Code
4XLNet (single model)89.7YesXLNet: Generalized Autoregressive Pretraining fo...2019-06-19Code
5T5-3B88.53YesExploring the Limits of Transfer Learning with a...2019-10-23Code
6T5-Large 770M86.66NoExploring the Limits of Transfer Learning with a...2019-10-23Code
7BERT-LARGE (Ensemble+TriviaQA)86.2NoBERT: Pre-training of Deep Bidirectional Transfo...2018-10-11Code
8T5-Base85.44YesExploring the Limits of Transfer Learning with a...2019-10-23Code
9BERT-LARGE (Single+TriviaQA)84.2NoBERT: Pre-training of Deep Bidirectional Transfo...2018-10-11Code
10BERT-Large-uncased-PruneOFA (90% unstruct sparse)83.35NoPrune Once for All: Sparse Pre-Trained Language ...2021-11-10Code
11BERT-Large-uncased-PruneOFA (90% unstruct sparse, QAT Int8)83.22NoPrune Once for All: Sparse Pre-Trained Language ...2021-11-10Code
12BERT-Base-uncased-PruneOFA (85% unstruct sparse)81.1NoPrune Once for All: Sparse Pre-Trained Language ...2021-11-10Code
13BERT-Base-uncased-PruneOFA (85% unstruct sparse, QAT Int8)80.84NoPrune Once for All: Sparse Pre-Trained Language ...2021-11-10Code
14BERT-Base-uncased-PruneOFA (90% unstruct sparse)79.83NoPrune Once for All: Sparse Pre-Trained Language ...2021-11-10Code
15TinyBERT-6 67M79.7NoTinyBERT: Distilling BERT for Natural Language U...2019-09-23Code
16T5-Small79.1YesExploring the Limits of Transfer Learning with a...2019-10-23Code
17R.M-Reader (single)78.9NoReinforced Mnemonic Reader for Machine Reading C...2017-05-08Code
18DensePhrases78.3NoLearning Dense Representations of Phrases at Scale2020-12-23Code
19DistilBERT-uncased-PruneOFA (85% unstruct sparse)78.1NoPrune Once for All: Sparse Pre-Trained Language ...2021-11-10Code
20DistilBERT77.7NoDistilBERT, a distilled version of BERT: smaller...2019-10-02Code
21DistilBERT-uncased-PruneOFA (85% unstruct sparse, QAT Int8)77.03NoPrune Once for All: Sparse Pre-Trained Language ...2021-11-10Code
22DistilBERT-uncased-PruneOFA (90% unstruct sparse)76.91NoPrune Once for All: Sparse Pre-Trained Language ...2021-11-10Code
23KAR76.7NoExplicit Utilization of General Knowledge in Mac...2018-09-10-
24SAN (single)76.235NoStochastic Answer Networks for Machine Reading C...2017-12-10Code
25DistilBERT-uncased-PruneOFA (90% unstruct sparse, QAT Int8)75.62NoPrune Once for All: Sparse Pre-Trained Language ...2021-11-10Code
26FusionNet75.3NoFusionNet: Fusing via Fully-Aware Attention with...2017-11-16Code
27QANet (data aug x3)75.1NoQANet: Combining Local Convolution with Global S...2018-04-23Code
28QANet (data aug x2)74.5NoQANet: Combining Local Convolution with Global S...2018-04-23Code
29DCN+ (single)74.5NoDCN+: Mixed Objective and Deep Residual Coattent...2017-10-31Code
30QANet73.6NoQANet: Combining Local Convolution with Global S...2018-04-23Code
31PhaseCond (single)72.1NoPhase Conductor on Multi-layered Attentions for ...2017-10-28-
32SRU71.4NoSimple Recurrent Units for Highly Parallelizable...2017-09-08Code
33Smarnet71.362NoSmarnet: Teaching Machines to Read and Comprehen...2017-10-08-
34DCN (Char + CoVe)71.3NoLearned in Translation: Contextualized Word Vect...2017-08-01Code
35R-NET (single)71.1No---
36Ruminating Reader70.6NoRuminating Reader: Reasoning with Gated Multi-Ho...2017-04-24-
37FastQAExt (beam-size 5)70.3NoMaking Neural QA as Simple as Possible but not S...2017-03-14Code
38DrQA (Document Reader only)69.5NoReading Wikipedia to Answer Open-Domain Questions2017-03-31Code
39jNet (TreeLSTM adaptation, QTLa, K=100)69.1NoExploring Question Understanding and Adaptation ...2017-03-14-
40SEDT-LSTM67.89NoStructural Embedding of Syntactic Trees for Mach...2017-03-02-
41BIDAF (single)67.7NoBidirectional Attention Flow for Machine Compreh...2016-11-05Code
42SECT-LSTM67.65NoStructural Embedding of Syntactic Trees for Mach...2017-03-02-
43RASOR66.4NoLearning Recurrent Span Representations for Extr...2016-11-04Code
44MPCM66.1NoMulti-Perspective Context Matching for Machine C...2016-12-13Code
45DCN65.4NoDynamic Coattention Networks For Question Answer...2016-11-05Code
46FABIR65.1NoA Fully Attention-Based Information Retriever2018-10-22Code
47Match-LSTM with Bi-Ans-Ptr (Boundary+Search+b) 64.1NoMachine Comprehension Using Match-LSTM and Answe...2016-08-29Code
48OTF dict+spelling (single)63.06NoLearning to Compute Word Embeddings On the Fly2017-06-01-
49DCR62.5NoEnd-to-End Answer Chunk Extraction and Ranking f...2016-10-31-
50FG fine-grained gate59.95NoWords or Characters? Fine-grained Gating for Rea...2016-11-06Code
51SPARTA59.3NoSPARTA: Efficient Open-Domain Question Answering...2020-09-28Code
52Blended RAG57.63NoBlended RAG: Improving RAG (Retriever-Augmented ...2024-03-22Code
53BERTserini50.2NoData Augmentation for BERT Fine-Tuning in Open-D...2019-04-14-
54BERTserini38.6NoEnd-to-End Open-Domain Question Answering with B...2019-02-05Code