Metric: F1 (higher is better)
| # | Model↕ | F1▼ | Extra Data | Paper | Date↕ | Code |
|---|---|---|---|---|---|---|
| 1 | {ANNA} (single model) | 95.719 | Yes | - | - | - |
| 2 | LUKE 483M | 95.4 | No | LUKE: Deep Contextualized Entity Representations... | 2020-10-02 | Code |
| 3 | LUKE (single model) | 95.379 | Yes | LUKE: Deep Contextualized Entity Representations... | 2020-10-02 | Code |
| 4 | LUKE (single model) | 95.379 | No | LUKE: Deep Contextualized Entity Representations... | 2020-10-02 | Code |
| 5 | XLNet (single model) | 95.08 | No | XLNet: Generalized Autoregressive Pretraining fo... | 2019-06-19 | Code |
| 6 | XLNet (single model) | 95.08 | Yes | XLNet: Generalized Autoregressive Pretraining fo... | 2019-06-19 | Code |
| 7 | XLNET-123 (single model) | 94.93 | No | - | - | - |
| 8 | XLNET-123++ (single model) | 94.903 | No | - | - | - |
| 9 | XLNET-123+ (single model) | 94.859 | No | - | - | - |
| 10 | SpanBERT (single model) | 94.635 | No | - | - | - |
| 11 | SpanBERT (single model) | 94.6 | No | SpanBERT: Improving Pre-training by Representing... | 2019-07-24 | Code |
| 12 | Unnamed submission by NMC | 94.584 | No | - | - | - |
| 13 | BERTSP (single model) | 94.584 | No | - | - | - |
| 14 | BERT+WWM+MT (single model) | 94.393 | No | - | - | - |
| 15 | Tuned BERT-1seq Large Cased (single model) | 93.294 | No | - | - | - |
| 16 | BERT-LARGE (Ensemble+TriviaQA) | 93.2 | No | BERT: Pre-training of Deep Bidirectional Transfo... | 2018-10-11 | Code |
| 17 | BERT (ensemble) | 93.16 | No | BERT: Pre-training of Deep Bidirectional Transfo... | 2018-10-11 | Code |
| 18 | BART (TextBox 2.0) | 93.04 | No | TextBox 2.0: A Text Generation Library with Pre-... | 2022-12-26 | Code |
| 19 | LinkBERT (large) | 92.7 | No | LinkBERT: Pretraining Language Models with Docum... | 2022-03-29 | Code |
| 20 | BERT+MT (single model) | 92.645 | No | - | - | - |
| 21 | ATB (single model) | 92.641 | No | - | - | - |
| 22 | Tuned BERT Large Cased (single model) | 92.617 | No | - | - | - |
| 23 | Knowledge-enhanced BERT (single model) | 92.425 | No | - | - | - |
| 24 | KT-NET (single model) | 92.425 | No | - | - | - |
| 25 | DPN (single model) | 92.019 | No | - | - | - |
| 26 | ST_bl | 91.976 | No | - | - | - |
| 27 | BERT-uncased (single model) | 91.932 | No | - | - | - |
| 28 | BERT (single model) | 91.835 | Yes | BERT: Pre-training of Deep Bidirectional Transfo... | 2018-10-11 | Code |
| 29 | EL-BERT (single model) | 91.807 | No | - | - | - |
| 30 | BERT-LARGE (Single+TriviaQA) | 91.8 | No | BERT: Pre-training of Deep Bidirectional Transfo... | 2018-10-11 | Code |
| 31 | BISAN (single model) | 91.756 | No | - | - | - |
| 32 | BERT+Sparse-Transformer | 91.623 | No | - | - | - |
| 33 | BERT-Large 32k batch size with AdamW | 91.58 | No | A Large Batch Optimizer Reality Check: Tradition... | 2021-02-12 | - |
| 34 | Original BERT Large Cased (single model) | 91.281 | No | - | - | - |
| 35 | nlnet (ensemble) | 91.202 | No | - | - | - |
| 36 | DyREX | 91.01 | No | DyREx: Dynamic Query Representation for Extracti... | 2022-10-26 | Code |
| 37 | Common-sense Governed BERT-123 (single model) | 90.613 | No | - | - | - |
| 38 | WD (single model) | 90.561 | No | - | - | - |
| 39 | WD1 (single model) | 90.429 | No | - | - | - |
| 40 | nlnet (single model) | 90.133 | No | - | - | - |
| 41 | MARS (ensemble) | 89.796 | No | - | - | - |
| 42 | BERT-Base mod (single model) | 89.379 | No | - | - | - |
| 43 | QANet (single) | 89.306 | No | - | - | - |
| 44 | Hybrid AoA Reader (ensemble) | 89.281 | No | - | - | - |
| 45 | Pytalk + Stanza + BERT (single model) | 89.218 | No | - | - | - |
| 46 | MMIPN | 88.948 | No | - | - | - |
| 47 | BERT (single model) | 88.947 | No | - | - | - |
| 48 | ARSG-BERT (single model) | 88.909 | No | - | - | - |
| 49 | Reinforced Mnemonic Reader + A2D (ensemble model) | 88.764 | No | - | - | - |
| 50 | SLQA+ (ensemble) | 88.607 | No | - | - | - |
| 51 | Reinforced Mnemonic Reader (ensemble model) | 88.533 | Yes | Reinforced Mnemonic Reader for Machine Reading C... | 2017-05-08 | Code |
| 52 | BERT - 6 Layers | 88.5 | No | Information Theoretic Representation Distillation | 2021-12-01 | Code |
| 53 | r-net+ (ensemble) | 88.493 | No | - | - | - |
| 54 | batch (single model) | 88.263 | No | - | - | - |
| 55 | mBERT + Task Adapter (Single) | 88.169 | No | - | - | - |
| 56 | AttentionReader+ (ensemble) | 88.163 | No | - | - | - |
| 57 | r-net (ensemble) | 88.126 | No | - | - | - |
| 58 | Reinforced Mnemonic Reader + A2D + DA (single model) | 88.122 | No | - | - | - |
| 59 | BERT-COMPOUND-DSS (single model) | 87.999 | No | - | - | - |
| 60 | BERT-COMPOUND (single model) | 87.758 | No | - | - | - |
| 61 | KACTEIL-MRC(GF-Net+) (ensemble) | 87.557 | No | - | - | - |
| 62 | Reinforced Mnemonic Reader + A2D (single model) | 87.454 | No | - | - | - |
| 63 | BiDAF + Self Attention + ELMo (ensemble) | 87.432 | No | Deep contextualized word representations | 2018-02-15 | Code |
| 64 | BiDAF + Self Attention + ELMo (ensemble) | 87.432 | No | Deep contextualized word representations | 2018-02-15 | - |
| 65 | BERT-INDEPENDENT-DSS-FILTERED (single model) | 87.374 | No | - | - | - |
| 66 | AVIQA+ (ensemble) | 87.311 | No | - | - | - |
| 67 | Hybrid AoA Reader (single model) | 87.288 | No | - | - | - |
| 68 | SLQA+ | 87.021 | No | - | - | - |
| 69 | {EAZI} (ensemble) | 86.912 | No | - | - | - |
| 70 | EAZI+ (ensemble) | 86.912 | No | - | - | - |
| 71 | MAMCN+ (single model) | 86.727 | No | - | - | - |
| 72 | MAMCN+ (single model) | 86.727 | No | - | - | - |
| 73 | DNET (ensemble) | 86.721 | No | - | - | - |
| 74 | BiDAF + Self Attention + ELMo + A2D (single model) | 86.711 | No | - | - | - |
| 75 | BERT-INDEPENDENT (single model) | 86.663 | No | - | - | - |
| 76 | Reinforced Mnemonic Reader (single model) | 86.654 | No | Reinforced Mnemonic Reader for Machine Reading C... | 2017-05-08 | Code |
| 77 | SLQA+ (single model) | 86.59 | No | - | - | - |
| 78 | r-net+ (single model) | 86.536 | No | - | - | - |
| 79 | SAN (ensemble model) | 86.496 | No | Stochastic Answer Networks for Machine Reading C... | 2017-12-10 | Code |
| 80 | Interactive AoA Reader+ (ensemble) | 86.45 | No | - | - | - |
| 81 | MIR-MRC(F-Net) (single model) | 86.288 | No | - | - | - |
| 82 | KACTEIL-MRC(GF-Net+Distillation) (single model) | 86.288 | No | - | - | - |
| 83 | KACTEIL-MRC (GF-Net+Distillation) | 86.288 | No | - | - | - |
| 84 | FusionNet (ensemble) | 86.016 | No | FusionNet: Fusing via Fully-Aware Attention with... | 2017-11-16 | Code |
| 85 | MDReader | 86.006 | No | - | - | - |
| 86 | DCN+ (ensemble) | 85.996 | No | DCN+: Mixed Objective and Deep Residual Coattent... | 2017-10-31 | Code |
| 87 | BiDAF + Self Attention + ELMo (single model) | 85.833 | No | Deep contextualized word representations | 2018-02-15 | Code |
| 88 | BiDAF + Self Attention + ELMo (single model) | 85.833 | No | Deep contextualized word representations | 2018-02-15 | - |
| 89 | BERT - 3 Layers | 85.8 | No | Information Theoretic Representation Distillation | 2021-12-01 | Code |
| 90 | KACTEIL-MRC(GF-Net+) (single model) | 85.78 | No | - | - | - |
| 91 | KACTEIL-MRC (GF-Net+) | 85.78 | No | - | - | - |
| 92 | KakaoNet (single model) | 85.724 | No | - | - | - |
| 93 | SLQA(ensemble) | 85.682 | No | - | - | - |
| 94 | SLQA (ensemble) | 85.682 | No | - | - | - |
| 95 | MDReader0 | 85.543 | No | - | - | - |
| 96 | BiDAF++ with pair2vec (single model) | 85.535 | No | - | - | - |
| 97 | aviqa (ensemble) | 85.469 | No | - | - | - |
| 98 | test | 85.348 | No | - | - | - |
| 99 | MEMEN (single model) | 85.344 | No | MEMEN: Multi-layer Embedding with Memory Network... | 2017-07-28 | - |
| 100 | MEMEN (single model) | 85.344 | No | MEMEN: Multi-layer Embedding with Memory Network... | 2017-07-28 | - |
| 101 | Interactive AoA Reader (ensemble) | 85.297 | No | - | - | - |
| 102 | AttentionReader+ (single) | 84.925 | No | - | - | - |
| 103 | DNET (single model) | 84.905 | No | - | - | - |
| 104 | BiDAF++ (single model) | 84.858 | No | - | - | - |
| 105 | MARS (single model) | 84.739 | No | - | - | - |
| 106 | Conductor-net (ensemble) | 84.63 | No | Phase Conductor on Multi-layered Attentions for ... | 2017-10-28 | - |
| 107 | QANet + data augmentation ×3 | 84.6 | No | QANet: Combining Local Convolution with Global S... | 2018-04-23 | Code |
| 108 | RuBERT | 84.6 | No | Adaptation of Deep Bidirectional Multilingual Tr... | 2019-05-17 | Code |
| 109 | FRC (single model) | 84.599 | No | - | - | - |
| 110 | VS^3-NET (single model) | 84.491 | No | - | - | - |
| 111 | Jenga (ensemble) | 84.466 | No | - | - | - |
| 112 | SAN (single model) | 84.396 | No | Stochastic Answer Networks for Machine Reading C... | 2017-12-10 | Code |
| 113 | r-net (single model) | 84.265 | No | - | - | - |
| 114 | r-net (single model) | 84.265 | No | - | - | - |
| 115 | RaSoR + TR + LM (single model) | 84.163 | No | Contextualized Word Representations for Reading ... | 2017-12-10 | Code |
| 116 | Conductor-net (ensemble) | 83.991 | No | - | - | - |
| 117 | {gqa} (single model) | 83.931 | No | - | - | - |
| 118 | FusionNet (single model) | 83.9 | No | FusionNet: Fusing via Fully-Aware Attention with... | 2017-11-16 | Code |
| 119 | Interactive AoA Reader+ (single model) | 83.843 | No | - | - | - |
| 120 | KAR (single model) | 83.538 | No | Explicit Utilization of General Knowledge in Mac... | 2018-09-10 | - |
| 121 | smarnet (ensemble) | 83.475 | No | - | - | - |
| 122 | Kbs (single model) | 83.405 | No | - | - | - |
| 123 | AVIQA-v2 (single model) | 83.305 | No | - | - | - |
| 124 | RaSoR + TR (single model) | 83.261 | No | Contextualized Word Representations for Reading ... | 2017-12-10 | Code |
| 125 | EfficientQA 125M | 83.1 | No | EfficientQA : a RoBERTa Based Phrase-Indexed Que... | 2021-01-06 | - |
| 126 | SLQA (single model) | 82.815 | No | - | - | - |
| 127 | DCN+ (single model) | 82.806 | No | DCN+: Mixed Objective and Deep Residual Coattent... | 2017-10-31 | Code |
| 128 | Mixed model (ensemble) | 82.769 | No | - | - | - |
| 129 | Conductor-net (single model) | 82.742 | No | Phase Conductor on Multi-layered Attentions for ... | 2017-10-28 | - |
| 130 | two-attention-self-attention (ensemble) | 82.716 | No | - | - | - |
| 131 | MEMEN (ensemble) | 82.658 | No | MEMEN: Multi-layer Embedding with Memory Network... | 2017-07-28 | - |
| 132 | ReasoNet (ensemble) | 82.552 | Yes | ReasoNet: Learning to Stop Reading in Machine Co... | 2016-09-17 | - |
| 133 | eeAttNet (single model) | 82.501 | No | - | - | - |
| 134 | Mnemonic Reader (ensemble) | 82.371 | No | Reinforced Mnemonic Reader for Machine Reading C... | 2017-05-08 | Code |
| 135 | S^3-Net (ensemble) | 82.342 | No | - | - | - |
| 136 | Conductor-net (single) | 81.933 | No | Phase Conductor on Multi-layered Attentions for ... | 2017-10-28 | - |
| 137 | Interactive AoA Reader (single model) | 81.931 | No | - | - | - |
| 138 | SEDT (ensemble model) | 81.761 | No | Structural Embedding of Syntactic Trees for Mach... | 2017-03-02 | - |
| 139 | Jenga (single model) | 81.754 | No | - | - | - |
| 140 | SSAE (ensemble) | 81.665 | No | - | - | - |
| 141 | SEDT+BiDAF (ensemble) | 81.53 | No | Structural Embedding of Syntactic Trees for Mach... | 2017-03-02 | - |
| 142 | BiDAF (ensemble) | 81.525 | No | Bidirectional Attention Flow for Machine Compreh... | 2016-11-05 | Code |
| 143 | jNet (ensemble) | 81.517 | No | Exploring Question Understanding and Adaptation ... | 2017-03-14 | - |
| 144 | Conductor-net (single) | 81.415 | No | - | - | - |
| 145 | Multi-Perspective Matching (ensemble) | 81.257 | No | Multi-Perspective Context Matching for Machine C... | 2016-12-13 | Code |
| 146 | BiDAF + Self Attention (single model) | 81.048 | No | Simple and Effective Multi-Paragraph Reading Com... | 2017-10-29 | Code |
| 147 | S^3-Net (single model) | 81.023 | No | - | - | - |
| 148 | two-attention-self-attention (single model) | 81.011 | No | - | - | - |
| 149 | T-gating (ensemble) | 81.001 | No | - | - | - |
| 150 | AVIQA (single model) | 80.55 | No | - | - | - |
| 151 | attention+self-attention (single model) | 80.462 | No | - | - | - |
| 152 | Dynamic Coattention Networks (ensemble) | 80.383 | No | Dynamic Coattention Networks For Question Answer... | 2016-11-05 | Code |
| 153 | SRU | 80.2 | No | Simple Recurrent Units for Highly Parallelizable... | 2017-09-08 | Code |
| 154 | smarnet (single model) | 80.16 | No | Smarnet: Teaching Machines to Read and Comprehen... | 2017-10-08 | - |
| 155 | Mnemonic Reader (single model) | 80.146 | No | Reinforced Mnemonic Reader for Machine Reading C... | 2017-05-08 | Code |
| 156 | QFASE | 79.989 | No | - | - | - |
| 157 | MAMCN (single model) | 79.939 | No | - | - | - |
| 158 | DCN + Char + CoVe | 79.9 | No | Learned in Translation: Contextualized Word Vect... | 2017-08-01 | Code |
| 159 | M-NET (single) | 79.835 | No | - | - | - |
| 160 | jNet (single model) | 79.821 | No | Exploring Question Understanding and Adaptation ... | 2017-03-14 | - |
| 161 | AttReader (single) | 79.725 | No | - | - | - |
| 162 | Ruminating Reader (single model) | 79.456 | No | Ruminating Reader: Reasoning with Gated Multi-Ho... | 2017-04-24 | - |
| 163 | ReasoNet (single model) | 79.364 | No | ReasoNet: Learning to Stop Reading in Machine Co... | 2016-09-17 | - |
| 164 | Document Reader (single model) | 79.353 | No | Reading Wikipedia to Answer Open-Domain Questions | 2017-03-31 | Code |
| 165 | FastQAExt | 78.857 | No | Making Neural QA as Simple as Possible but not S... | 2017-03-14 | Code |
| 166 | Multi-Perspective Matching (single model) | 78.784 | No | Multi-Perspective Context Matching for Machine C... | 2016-12-13 | Code |
| 167 | RaSoR (single model) | 78.741 | No | Learning Recurrent Span Representations for Extr... | 2016-11-04 | Code |
| 168 | SSR-BiDAF | 78.358 | No | - | - | - |
| 169 | SimpleBaseline (single model) | 78.236 | No | - | - | - |
| 170 | SEDT+BiDAF (single model) | 77.971 | No | Structural Embedding of Syntactic Trees for Mach... | 2017-03-02 | - |
| 171 | PQMN (single model) | 77.783 | No | - | - | - |
| 172 | FABIR | 77.605 | No | A Fully Attention-Based Information Retriever | 2018-10-22 | Code |
| 173 | T-gating (single model) | 77.569 | No | - | - | - |
| 174 | SEDT (single model) | 77.527 | No | Structural Embedding of Syntactic Trees for Mach... | 2017-03-02 | - |
| 175 | BiDAF (single model) | 77.323 | No | Bidirectional Attention Flow for Machine Compreh... | 2016-11-05 | Code |
| 176 | AllenNLP BiDAF (single model) | 77.151 | No | - | - | - |
| 177 | FastQA | 77.07 | No | Making Neural QA as Simple as Possible but not S... | 2017-03-14 | Code |
| 178 | Match-LSTM with Ans-Ptr (Boundary) (ensemble) | 77.022 | No | Machine Comprehension Using Match-LSTM and Answe... | 2016-08-29 | Code |
| 179 | Iterative Co-attention Network | 76.786 | No | - | - | - |
| 180 | BIDAF-COMPOUND-DSS (single model) | 76.429 | No | - | - | - |
| 181 | BIDAF-INDEPENDENT-DSS (single model) | 76.349 | No | - | - | - |
| 182 | Dynamic Coattention Networks (single model) | 75.896 | No | Dynamic Coattention Networks For Question Answer... | 2016-11-05 | Code |
| 183 | newtest | 75.787 | No | - | - | - |
| 184 | BIDAF-INDEPENDENT (single model) | 74.594 | No | - | - | - |
| 185 | BIDAF-COMPOUND (single model) | 74.555 | No | - | - | - |
| 186 | Unnamed submission by ravioncodalab | 73.921 | No | - | - | - |
| 187 | Match-LSTM with Bi-Ans-Ptr (Boundary) | 73.743 | No | Machine Comprehension Using Match-LSTM and Answe... | 2016-08-29 | Code |
| 188 | Attentive CNN context with LSTM | 73.463 | No | - | - | - |
| 189 | Fine-Grained Gating | 73.327 | No | Words or Characters? Fine-grained Gating for Rea... | 2016-11-06 | Code |
| 190 | OTF dict+spelling (single) | 73.056 | No | Learning to Compute Word Embeddings On the Fly | 2017-06-01 | - |
| 191 | OTF spelling (single) | 72.016 | No | Learning to Compute Word Embeddings On the Fly | 2017-06-01 | - |
| 192 | OTF spelling+lemma (single) | 71.968 | No | Learning to Compute Word Embeddings On the Fly | 2017-06-01 | - |
| 193 | RQA+IDR (single model) | 71.389 | No | Harvesting and Refining Question-Answer Pairs fo... | 2020-05-06 | Code |
| 194 | RQA+IDR (single model) | 71.389 | No | Harvesting and Refining Question-Answer Pairs fo... | 2020-05-06 | Code |
| 195 | Dynamic Chunk Reader | 70.956 | No | End-to-End Answer Chunk Extraction and Ranking f... | 2016-10-31 | - |
| 196 | Match-LSTM with Ans-Ptr (Boundary) | 70.695 | No | Machine Comprehension Using Match-LSTM and Answe... | 2016-08-29 | Code |
| 197 | Unnamed submission by Will_Wu | 69.436 | No | - | - | - |
| 198 | Match-LSTM with Ans-Ptr (Sentence) | 67.748 | No | Machine Comprehension Using Match-LSTM and Answe... | 2016-08-29 | Code |
| 199 | RQA (single model) | 65.467 | No | Harvesting and Refining Question-Answer Pairs fo... | 2020-05-06 | Code |
| 200 | RQA (single model) | 65.467 | No | Harvesting and Refining Question-Answer Pairs fo... | 2020-05-06 | Code |
| 201 | UQA (single model) | 64.036 | No | - | - | - |
| 202 | Unnamed submission by jinhyuklee | 62.78 | No | - | - | - |
| 203 | Unnamed submission by minjoon | 62.757 | No | - | - | - |
| 204 | UnsupervisedQA V1 (ensemble) | 56.436 | No | - | - | - |
| 205 | UnsupervisedQA V1 (single model) | 54.723 | No | - | - | - |
| 206 | QANet (single model) | 13.211 | No | - | - | - |
| 207 | 6.907 | No | - | - | - | |
| 208 | QANet (ensemble) | 0 | No | - | - | - |
| 209 | superman-new-des | 0 | No | - | - | - |
| 210 | WAHnGREA | 0 | No | - | - | - |
| 211 | superman-des | 0 | No | - | - | - |
| 212 | XLNet-deep (ensemble) | 0 | No | - | - | - |