Metric: EM (higher is better)
| # | Model↕ | EM▼ | Extra Data | Paper | Date↕ | Code |
|---|---|---|---|---|---|---|
| 1 | {ANNA} (single model) | 90.622 | Yes | - | - | - |
| 2 | LUKE (single model) | 90.202 | Yes | LUKE: Deep Contextualized Entity Representations... | 2020-10-02 | Code |
| 3 | LUKE (single model) | 90.202 | No | LUKE: Deep Contextualized Entity Representations... | 2020-10-02 | Code |
| 4 | LUKE | 90.2 | No | LUKE: Deep Contextualized Entity Representations... | 2020-10-02 | Code |
| 5 | XLNet (single model) | 89.898 | No | XLNet: Generalized Autoregressive Pretraining fo... | 2019-06-19 | Code |
| 6 | XLNet (single model) | 89.898 | Yes | XLNet: Generalized Autoregressive Pretraining fo... | 2019-06-19 | Code |
| 7 | XLNET-123++ (single model) | 89.856 | No | - | - | - |
| 8 | XLNET-123+ (single model) | 89.709 | No | - | - | - |
| 9 | XLNET-123 (single model) | 89.646 | No | - | - | - |
| 10 | Unnamed submission by NMC | 88.912 | No | - | - | - |
| 11 | BERTSP (single model) | 88.912 | No | - | - | - |
| 12 | SpanBERT (single model) | 88.839 | No | - | - | - |
| 13 | SpanBERT (single model) | 88.8 | No | SpanBERT: Improving Pre-training by Representing... | 2019-07-24 | Code |
| 14 | BERT+WWM+MT (single model) | 88.65 | No | - | - | - |
| 15 | Tuned BERT-1seq Large Cased (single model) | 87.465 | No | - | - | - |
| 16 | LinkBERT (large) | 87.45 | No | LinkBERT: Pretraining Language Models with Docum... | 2022-03-29 | Code |
| 17 | BERT (ensemble) | 87.433 | No | BERT: Pre-training of Deep Bidirectional Transfo... | 2018-10-11 | Code |
| 18 | BERT-LARGE (Ensemble+TriviaQA) | 87.4 | No | BERT: Pre-training of Deep Bidirectional Transfo... | 2018-10-11 | Code |
| 19 | ATB (single model) | 86.94 | No | - | - | - |
| 20 | Tuned BERT Large Cased (single model) | 86.521 | No | - | - | - |
| 21 | BERT+MT (single model) | 86.458 | No | - | - | - |
| 22 | Knowledge-enhanced BERT (single model) | 85.944 | No | - | - | - |
| 23 | KT-NET (single model) | 85.944 | No | - | - | - |
| 24 | ST_bl | 85.43 | No | - | - | - |
| 25 | nlnet (ensemble) | 85.356 | No | - | - | - |
| 26 | EL-BERT (single model) | 85.335 | No | - | - | - |
| 27 | BISAN (single model) | 85.314 | No | - | - | - |
| 28 | BERT+Sparse-Transformer | 85.125 | No | - | - | - |
| 29 | BERT (single model) | 85.083 | Yes | BERT: Pre-training of Deep Bidirectional Transfo... | 2018-10-11 | Code |
| 30 | DPN (single model) | 84.978 | No | - | - | - |
| 31 | BERT-uncased (single model) | 84.926 | No | - | - | - |
| 32 | WD (single model) | 84.402 | No | - | - | - |
| 33 | Original BERT Large Cased (single model) | 84.328 | No | - | - | - |
| 34 | MARS (ensemble) | 83.982 | No | - | - | - |
| 35 | Common-sense Governed BERT-123 (single model) | 83.93 | No | - | - | - |
| 36 | WD1 (single model) | 83.804 | No | - | - | - |
| 37 | nlnet (single model) | 83.468 | No | - | - | - |
| 38 | Pytalk + Stanza + BERT (single model) | 83.426 | No | - | - | - |
| 39 | Reinforced Mnemonic Reader + A2D (ensemble model) | 82.849 | No | - | - | - |
| 40 | BERT-Base mod (single model) | 82.681 | No | - | - | - |
| 41 | r-net+ (ensemble) | 82.65 | No | - | - | - |
| 42 | Hybrid AoA Reader (ensemble) | 82.482 | No | - | - | - |
| 43 | QANet (single) | 82.471 | No | - | - | - |
| 44 | SLQA+ (ensemble) | 82.44 | No | - | - | - |
| 45 | Reinforced Mnemonic Reader (ensemble model) | 82.283 | Yes | Reinforced Mnemonic Reader for Machine Reading C... | 2017-05-08 | Code |
| 46 | r-net (ensemble) | 82.136 | No | - | - | - |
| 47 | BERT (single model) | 82.062 | No | - | - | - |
| 48 | AttentionReader+ (ensemble) | 81.79 | No | - | - | - |
| 49 | MMIPN | 81.58 | No | - | - | - |
| 50 | BERT - 6 Layers | 81.5 | No | Information Theoretic Representation Distillation | 2021-12-01 | Code |
| 51 | KACTEIL-MRC(GF-Net+) (ensemble) | 81.496 | No | - | - | - |
| 52 | Reinforced Mnemonic Reader + A2D + DA (single model) | 81.401 | No | - | - | - |
| 53 | ARSG-BERT (single model) | 81.307 | No | - | - | - |
| 54 | BERT-COMPOUND-DSS (single model) | 81.045 | No | - | - | - |
| 55 | BiDAF + Self Attention + ELMo (ensemble) | 81.003 | No | Deep contextualized word representations | 2018-02-15 | Code |
| 56 | BiDAF + Self Attention + ELMo (ensemble) | 81.003 | No | Deep contextualized word representations | 2018-02-15 | - |
| 57 | BERT-COMPOUND (single model) | 80.72 | No | - | - | - |
| 58 | mBERT + Task Adapter (Single) | 80.667 | No | - | - | - |
| 59 | AVIQA+ (ensemble) | 80.615 | No | - | - | - |
| 60 | Reinforced Mnemonic Reader + A2D (single model) | 80.489 | No | - | - | - |
| 61 | SLQA+ | 80.436 | No | - | - | - |
| 62 | {EAZI} (ensemble) | 80.436 | No | - | - | - |
| 63 | EAZI+ (ensemble) | 80.426 | No | - | - | - |
| 64 | DNET (ensemble) | 80.164 | No | - | - | - |
| 65 | Hybrid AoA Reader (single model) | 80.027 | No | - | - | - |
| 66 | BiDAF + Self Attention + ELMo + A2D (single model) | 79.996 | No | - | - | - |
| 67 | r-net+ (single model) | 79.901 | No | - | - | - |
| 68 | batch (single model) | 79.859 | No | - | - | - |
| 69 | MAMCN+ (single model) | 79.692 | No | - | - | - |
| 70 | MAMCN+ (single model) | 79.692 | No | - | - | - |
| 71 | SAN (ensemble model) | 79.608 | No | Stochastic Answer Networks for Machine Reading C... | 2017-12-10 | Code |
| 72 | BERT-INDEPENDENT-DSS-FILTERED (single model) | 79.597 | No | - | - | - |
| 73 | Reinforced Mnemonic Reader (single model) | 79.545 | No | Reinforced Mnemonic Reader for Machine Reading C... | 2017-05-08 | Code |
| 74 | SLQA+ (single model) | 79.199 | No | - | - | - |
| 75 | Interactive AoA Reader+ (ensemble) | 79.083 | No | - | - | - |
| 76 | MIR-MRC(F-Net) (single model) | 79.083 | No | - | - | - |
| 77 | KACTEIL-MRC(GF-Net+Distillation) (single model) | 79.083 | No | - | - | - |
| 78 | KACTEIL-MRC (GF-Net+Distillation) | 79.083 | No | - | - | - |
| 79 | MDReader | 79.031 | No | - | - | - |
| 80 | FusionNet (ensemble) | 78.978 | No | FusionNet: Fusing via Fully-Aware Attention with... | 2017-11-16 | Code |
| 81 | DCN+ (ensemble) | 78.852 | No | DCN+: Mixed Objective and Deep Residual Coattent... | 2017-10-31 | Code |
| 82 | KACTEIL-MRC(GF-Net+) (single model) | 78.664 | No | - | - | - |
| 83 | KACTEIL-MRC (GF-Net+) | 78.664 | No | - | - | - |
| 84 | BERT-INDEPENDENT (single model) | 78.653 | No | - | - | - |
| 85 | BiDAF + Self Attention + ELMo (single model) | 78.58 | No | Deep contextualized word representations | 2018-02-15 | Code |
| 86 | BiDAF + Self Attention + ELMo (single model) | 78.58 | No | Deep contextualized word representations | 2018-02-15 | - |
| 87 | aviqa (ensemble) | 78.496 | No | - | - | - |
| 88 | KakaoNet (single model) | 78.401 | No | - | - | - |
| 89 | SLQA(ensemble) | 78.328 | No | - | - | - |
| 90 | SLQA (ensemble) | 78.328 | No | - | - | - |
| 91 | MEMEN (single model) | 78.234 | No | MEMEN: Multi-layer Embedding with Memory Network... | 2017-07-28 | - |
| 92 | MEMEN (single model) | 78.234 | No | MEMEN: Multi-layer Embedding with Memory Network... | 2017-07-28 | - |
| 93 | BiDAF++ with pair2vec (single model) | 78.223 | No | - | - | - |
| 94 | MDReader0 | 78.171 | No | - | - | - |
| 95 | test | 78.087 | No | - | - | - |
| 96 | Interactive AoA Reader (ensemble) | 77.845 | No | - | - | - |
| 97 | BERT - 3 Layers | 77.7 | No | Information Theoretic Representation Distillation | 2021-12-01 | Code |
| 98 | DNET (single model) | 77.646 | No | - | - | - |
| 99 | RaSoR + TR + LM (single model) | 77.583 | No | Contextualized Word Representations for Reading ... | 2017-12-10 | Code |
| 100 | BiDAF++ (single model) | 77.573 | No | - | - | - |
| 101 | AttentionReader+ (single) | 77.342 | No | - | - | - |
| 102 | Jenga (ensemble) | 77.237 | No | - | - | - |
| 103 | {gqa} (single model) | 77.09 | No | - | - | - |
| 104 | Conductor-net (ensemble) | 76.996 | No | Phase Conductor on Multi-layered Attentions for ... | 2017-10-28 | - |
| 105 | MARS (single model) | 76.859 | No | - | - | - |
| 106 | SAN (single model) | 76.828 | No | Stochastic Answer Networks for Machine Reading C... | 2017-12-10 | Code |
| 107 | VS^3-NET (single model) | 76.775 | No | - | - | - |
| 108 | r-net (single model) | 76.461 | No | - | - | - |
| 109 | r-net (single model) | 76.461 | No | - | - | - |
| 110 | FRC (single model) | 76.24 | No | - | - | - |
| 111 | QANet + data augmentation ×3 | 76.2 | No | QANet: Combining Local Convolution with Global S... | 2018-04-23 | Code |
| 112 | Conductor-net (ensemble) | 76.146 | No | - | - | - |
| 113 | KAR (single model) | 76.125 | No | Explicit Utilization of General Knowledge in Mac... | 2018-09-10 | - |
| 114 | smarnet (ensemble) | 75.989 | No | - | - | - |
| 115 | FusionNet (single model) | 75.968 | No | FusionNet: Fusing via Fully-Aware Attention with... | 2017-11-16 | Code |
| 116 | AVIQA-v2 (single model) | 75.926 | No | - | - | - |
| 117 | Interactive AoA Reader+ (single model) | 75.821 | No | - | - | - |
| 118 | RaSoR + TR (single model) | 75.789 | No | Contextualized Word Representations for Reading ... | 2017-12-10 | Code |
| 119 | MEMEN (ensemble) | 75.37 | No | MEMEN: Multi-layer Embedding with Memory Network... | 2017-07-28 | - |
| 120 | Mixed model (ensemble) | 75.265 | No | - | - | - |
| 121 | two-attention-self-attention (ensemble) | 75.223 | No | - | - | - |
| 122 | Kbs (single model) | 75.034 | No | - | - | - |
| 123 | ReasoNet (ensemble) | 75.034 | Yes | ReasoNet: Learning to Stop Reading in Machine Co... | 2016-09-17 | - |
| 124 | EfficientQA 125M | 74.9 | No | EfficientQA : a RoBERTa Based Phrase-Indexed Que... | 2021-01-06 | - |
| 125 | DCN+ (single model) | 74.866 | No | DCN+: Mixed Objective and Deep Residual Coattent... | 2017-10-31 | Code |
| 126 | eeAttNet (single model) | 74.604 | No | - | - | - |
| 127 | SLQA (single model) | 74.489 | No | - | - | - |
| 128 | Conductor-net (single model) | 74.405 | No | Phase Conductor on Multi-layered Attentions for ... | 2017-10-28 | - |
| 129 | Mnemonic Reader (ensemble) | 74.268 | No | Reinforced Mnemonic Reader for Machine Reading C... | 2017-05-08 | Code |
| 130 | S^3-Net (ensemble) | 74.121 | No | - | - | - |
| 131 | SEDT (ensemble model) | 74.09 | No | Structural Embedding of Syntactic Trees for Mach... | 2017-03-02 | - |
| 132 | SSAE (ensemble) | 74.08 | No | - | - | - |
| 133 | Multi-Perspective Matching (ensemble) | 73.765 | No | Multi-Perspective Context Matching for Machine C... | 2016-12-13 | Code |
| 134 | BiDAF (ensemble) | 73.744 | No | Bidirectional Attention Flow for Machine Compreh... | 2016-11-05 | Code |
| 135 | SEDT+BiDAF (ensemble) | 73.723 | No | Structural Embedding of Syntactic Trees for Mach... | 2017-03-02 | - |
| 136 | Interactive AoA Reader (single model) | 73.639 | No | - | - | - |
| 137 | Jenga (single model) | 73.303 | No | - | - | - |
| 138 | Conductor-net (single) | 73.24 | No | Phase Conductor on Multi-layered Attentions for ... | 2017-10-28 | - |
| 139 | jNet (ensemble) | 73.01 | No | Exploring Question Understanding and Adaptation ... | 2017-03-14 | - |
| 140 | T-gating (ensemble) | 72.758 | No | - | - | - |
| 141 | two-attention-self-attention (single model) | 72.6 | No | - | - | - |
| 142 | Conductor-net (single) | 72.59 | No | - | - | - |
| 143 | AVIQA (single model) | 72.485 | No | - | - | - |
| 144 | BiDAF + Self Attention (single model) | 72.139 | No | Simple and Effective Multi-Paragraph Reading Com... | 2017-10-29 | Code |
| 145 | S^3-Net (single model) | 71.908 | No | - | - | - |
| 146 | QFASE | 71.898 | No | - | - | - |
| 147 | attention+self-attention (single model) | 71.698 | No | - | - | - |
| 148 | Dynamic Coattention Networks (ensemble) | 71.625 | No | Dynamic Coattention Networks For Question Answer... | 2016-11-05 | Code |
| 149 | smarnet (single model) | 71.415 | No | Smarnet: Teaching Machines to Read and Comprehen... | 2017-10-08 | - |
| 150 | SRU | 71.4 | No | Simple Recurrent Units for Highly Parallelizable... | 2017-09-08 | Code |
| 151 | AttReader (single) | 71.373 | No | - | - | - |
| 152 | DCN + Char + CoVe | 71.3 | No | Learned in Translation: Contextualized Word Vect... | 2017-08-01 | Code |
| 153 | M-NET (single) | 71.016 | No | - | - | - |
| 154 | Mnemonic Reader (single model) | 70.995 | No | Reinforced Mnemonic Reader for Machine Reading C... | 2017-05-08 | Code |
| 155 | MAMCN (single model) | 70.985 | No | - | - | - |
| 156 | FastQAExt | 70.849 | No | Making Neural QA as Simple as Possible but not S... | 2017-03-14 | Code |
| 157 | RaSoR (single model) | 70.849 | No | Learning Recurrent Span Representations for Extr... | 2016-11-04 | Code |
| 158 | Document Reader (single model) | 70.733 | No | Reading Wikipedia to Answer Open-Domain Questions | 2017-03-31 | Code |
| 159 | Ruminating Reader (single model) | 70.639 | No | Ruminating Reader: Reasoning with Gated Multi-Ho... | 2017-04-24 | - |
| 160 | jNet (single model) | 70.607 | No | Exploring Question Understanding and Adaptation ... | 2017-03-14 | - |
| 161 | ReasoNet (single model) | 70.555 | No | ReasoNet: Learning to Stop Reading in Machine Co... | 2016-09-17 | - |
| 162 | Multi-Perspective Matching (single model) | 70.387 | No | Multi-Perspective Context Matching for Machine C... | 2016-12-13 | Code |
| 163 | DrQA | 70 | No | Reading Wikipedia to Answer Open-Domain Questions | 2017-03-31 | Code |
| 164 | SimpleBaseline (single model) | 69.6 | No | - | - | - |
| 165 | SSR-BiDAF | 69.443 | No | - | - | - |
| 166 | SEDT+BiDAF (single model) | 68.478 | No | Structural Embedding of Syntactic Trees for Mach... | 2017-03-02 | - |
| 167 | FastQA | 68.436 | No | Making Neural QA as Simple as Possible but not S... | 2017-03-14 | Code |
| 168 | PQMN (single model) | 68.331 | No | - | - | - |
| 169 | SEDT (single model) | 68.163 | No | Structural Embedding of Syntactic Trees for Mach... | 2017-03-02 | - |
| 170 | T-gating (single model) | 68.132 | No | - | - | - |
| 171 | BiDAF (single model) | 67.974 | No | Bidirectional Attention Flow for Machine Compreh... | 2016-11-05 | Code |
| 172 | Match-LSTM with Ans-Ptr (Boundary) (ensemble) | 67.901 | No | Machine Comprehension Using Match-LSTM and Answe... | 2016-08-29 | Code |
| 173 | FABIR | 67.744 | No | A Fully Attention-Based Information Retriever | 2018-10-22 | Code |
| 174 | AllenNLP BiDAF (single model) | 67.618 | No | - | - | - |
| 175 | BIDAF-COMPOUND-DSS (single model) | 67.544 | No | - | - | - |
| 176 | Iterative Co-attention Network | 67.502 | No | - | - | - |
| 177 | newtest | 66.527 | No | - | - | - |
| 178 | BIDAF-INDEPENDENT-DSS (single model) | 66.516 | No | - | - | - |
| 179 | Dynamic Coattention Networks (single model) | 66.233 | No | Dynamic Coattention Networks For Question Answer... | 2016-11-05 | Code |
| 180 | DCN | 66.2 | No | Dynamic Coattention Networks For Question Answer... | 2016-11-05 | Code |
| 181 | MPCM | 65.5 | No | Multi-Perspective Context Matching for Machine C... | 2016-12-13 | Code |
| 182 | BIDAF-COMPOUND (single model) | 65.163 | No | - | - | - |
| 183 | BIDAF-INDEPENDENT (single model) | 64.932 | No | - | - | - |
| 184 | Match-LSTM with Bi-Ans-Ptr (Boundary) | 64.744 | No | Machine Comprehension Using Match-LSTM and Answe... | 2016-08-29 | Code |
| 185 | Unnamed submission by ravioncodalab | 64.439 | No | - | - | - |
| 186 | OTF dict+spelling (single) | 64.083 | No | Learning to Compute Word Embeddings On the Fly | 2017-06-01 | - |
| 187 | Attentive CNN context with LSTM | 63.306 | No | - | - | - |
| 188 | OTF spelling (single) | 62.897 | No | Learning to Compute Word Embeddings On the Fly | 2017-06-01 | - |
| 189 | OTF spelling+lemma (single) | 62.604 | No | Learning to Compute Word Embeddings On the Fly | 2017-06-01 | - |
| 190 | Dynamic Chunk Reader | 62.499 | No | End-to-End Answer Chunk Extraction and Ranking f... | 2016-10-31 | - |
| 191 | Fine-Grained Gating | 62.446 | No | Words or Characters? Fine-grained Gating for Rea... | 2016-11-06 | Code |
| 192 | RQA+IDR (single model) | 61.145 | No | Harvesting and Refining Question-Answer Pairs fo... | 2020-05-06 | Code |
| 193 | RQA+IDR (single model) | 61.145 | No | Harvesting and Refining Question-Answer Pairs fo... | 2020-05-06 | Code |
| 194 | Match-LSTM with Ans-Ptr (Boundary) | 60.474 | No | Machine Comprehension Using Match-LSTM and Answe... | 2016-08-29 | Code |
| 195 | Unnamed submission by Will_Wu | 59.058 | No | - | - | - |
| 196 | RQA (single model) | 55.827 | No | Harvesting and Refining Question-Answer Pairs fo... | 2020-05-06 | Code |
| 197 | RQA (single model) | 55.827 | No | Harvesting and Refining Question-Answer Pairs fo... | 2020-05-06 | Code |
| 198 | Match-LSTM with Ans-Ptr (Sentence) | 54.505 | No | Machine Comprehension Using Match-LSTM and Answe... | 2016-08-29 | Code |
| 199 | UQA (single model) | 53.698 | No | - | - | - |
| 200 | Unnamed submission by jinhyuklee | 52.544 | No | - | - | - |
| 201 | Unnamed submission by minjoon | 52.533 | No | - | - | - |
| 202 | UnsupervisedQA V1 (ensemble) | 47.341 | No | - | - | - |
| 203 | UnsupervisedQA V1 (single model) | 44.215 | No | - | - | - |
| 204 | QANet (single model) | 12.273 | No | - | - | - |
| 205 | 0 | No | - | - | - | |
| 206 | QANet (ensemble) | 0 | No | - | - | - |
| 207 | superman-new-des | 0 | No | - | - | - |
| 208 | WAHnGREA | 0 | No | - | - | - |
| 209 | superman-des | 0 | No | - | - | - |
| 210 | XLNet-deep (ensemble) | 0 | No | - | - | - |