TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

SotA/Natural Language Processing/Question Answering/SQuAD1.1 dev

Question Answering on SQuAD1.1 dev

Metric: F1 (higher is better)

LeaderboardDataset
Loading chart...

Results

Submit a result
#Model↕F1▼Extra DataPaperDate↕Code
1XLNet+DSC95.77YesDice Loss for Data-imbalanced NLP Tasks2019-11-07Code
2T5-11B95.64YesExploring the Limits of Transfer Learning with a...2019-10-23Code
3XLNet (single model)95.1YesXLNet: Generalized Autoregressive Pretraining fo...2019-06-19Code
4LUKE 483M95NoLUKE: Deep Contextualized Entity Representations...2020-10-02Code
5T5-3B94.95YesExploring the Limits of Transfer Learning with a...2019-10-23Code
6T5-Large 770M93.79NoExploring the Limits of Transfer Learning with a...2019-10-23Code
7BERT-LARGE (Ensemble+TriviaQA)92.2NoBERT: Pre-training of Deep Bidirectional Transfo...2018-10-11Code
8T5-Base92.08YesExploring the Limits of Transfer Learning with a...2019-10-23Code
9BERT-LARGE (Single+TriviaQA)91.1NoBERT: Pre-training of Deep Bidirectional Transfo...2018-10-11Code
10BART Base (with text infilling)90.8NoBART: Denoising Sequence-to-Sequence Pre-trainin...2019-10-29Code
11BERT large (LAMB optimizer)90.584NoLarge Batch Optimization for Deep Learning: Trai...2019-04-01Code
12BERT-Large-uncased-PruneOFA (90% unstruct sparse)90.2NoPrune Once for All: Sparse Pre-Trained Language ...2021-11-10Code
13BERT-Large-uncased-PruneOFA (90% unstruct sparse, QAT Int8)90.02NoPrune Once for All: Sparse Pre-Trained Language ...2021-11-10Code
14BERT-Base-uncased-PruneOFA (85% unstruct sparse)88.42NoPrune Once for All: Sparse Pre-Trained Language ...2021-11-10Code
15BERT-Base-uncased-PruneOFA (85% unstruct sparse, QAT Int8)88.24NoPrune Once for All: Sparse Pre-Trained Language ...2021-11-10Code
16TinyBERT-6 67M87.5NoTinyBERT: Distilling BERT for Natural Language U...2019-09-23Code
17BERT-Base-uncased-PruneOFA (90% unstruct sparse)87.25NoPrune Once for All: Sparse Pre-Trained Language ...2021-11-10Code
18T5-Small87.24YesExploring the Limits of Transfer Learning with a...2019-10-23Code
19R.M-Reader (single)86.3NoReinforced Mnemonic Reader for Machine Reading C...2017-05-08Code
20DensePhrases86.3NoLearning Dense Representations of Phrases at Scale2020-12-23Code
21DistilBERT-uncased-PruneOFA (85% unstruct sparse)85.82NoPrune Once for All: Sparse Pre-Trained Language ...2021-11-10Code
22DistilBERT 66M85.8NoDistilBERT, a distilled version of BERT: smaller...2019-10-02Code
23BiDAF + Self Attention + ELMo85.6NoDeep contextualized word representations2018-02-15Code
24DistilBERT-uncased-PruneOFA (85% unstruct sparse, QAT Int8)85.13NoPrune Once for All: Sparse Pre-Trained Language ...2021-11-10Code
25KAR84.9NoExplicit Utilization of General Knowledge in Mac...2018-09-10-
26DistilBERT-uncased-PruneOFA (90% unstruct sparse)84.82NoPrune Once for All: Sparse Pre-Trained Language ...2021-11-10Code
27SAN (single)84.056NoStochastic Answer Networks for Machine Reading C...2017-12-10Code
28DistilBERT-uncased-PruneOFA (90% unstruct sparse, QAT Int8)83.87NoPrune Once for All: Sparse Pre-Trained Language ...2021-11-10Code
29QANet (data aug x3)83.8NoQANet: Combining Local Convolution with Global S...2018-04-23Code
30FusionNet83.6NoFusionNet: Fusing via Fully-Aware Attention with...2017-11-16Code
31QANet (data aug x2)83.2NoQANet: Combining Local Convolution with Global S...2018-04-23Code
32DCN+ (single)83.1NoDCN+: Mixed Objective and Deep Residual Coattent...2017-10-31Code
33QANet82.7NoQANet: Combining Local Convolution with Global S...2018-04-23Code
34PhaseCond (single)81.4NoPhase Conductor on Multi-layered Attentions for ...2017-10-28-
35SRU80.2NoSimple Recurrent Units for Highly Parallelizable...2017-09-08Code
36Smarnet80.183NoSmarnet: Teaching Machines to Read and Comprehen...2017-10-08-
37DCN (Char + CoVe)79.9NoLearned in Translation: Contextualized Word Vect...2017-08-01Code
38R-NET (single)79.5No---
39Ruminating Reader79.5NoRuminating Reader: Reasoning with Gated Multi-Ho...2017-04-24-
40DrQA (Document Reader only)78.8NoReading Wikipedia to Answer Open-Domain Questions2017-03-31Code
41FastQAExt (beam-size 5)78.5NoMaking Neural QA as Simple as Possible but not S...2017-03-14Code
42jNet (TreeLSTM adaptation, QTLa, K=100)78.38NoExploring Question Understanding and Adaptation ...2017-03-14-
43SEDT-LSTM77.42NoStructural Embedding of Syntactic Trees for Mach...2017-03-02-
44BIDAF (single)77.3NoBidirectional Attention Flow for Machine Compreh...2016-11-05Code
45SECT-LSTM77.19NoStructural Embedding of Syntactic Trees for Mach...2017-03-02-
46MPCM75.8NoMulti-Perspective Context Matching for Machine C...2016-12-13Code
47DCN75.6NoDynamic Coattention Networks For Question Answer...2016-11-05Code
48FABIR75.6NoA Fully Attention-Based Information Retriever2018-10-22Code
49RASOR74.9NoLearning Recurrent Span Representations for Extr...2016-11-04Code
50FG fine-grained gate71.25NoWords or Characters? Fine-grained Gating for Rea...2016-11-06Code
51DCR71.2NoEnd-to-End Answer Chunk Extraction and Ranking f...2016-10-31-
52Match-LSTM with Bi-Ans-Ptr (Boundary+Search+b) 64.7NoMachine Comprehension Using Match-LSTM and Answe...2016-08-29Code