Danqi Chen, Adam Fisch, Jason Weston, Antoine Bordes
This paper proposes to tackle open- domain question answering using Wikipedia as the unique knowledge source: the answer to any factoid question is a text span in a Wikipedia article. This task of machine reading at scale combines the challenges of document retrieval (finding the relevant articles) with that of machine comprehension of text (identifying the answer spans from those articles). Our approach combines a search component based on bigram hashing and TF-IDF matching with a multi-layer recurrent neural network model trained to detect answers in Wikipedia paragraphs. Our experiments on multiple existing QA datasets indicate that (1) both modules are highly competitive with respect to existing counterparts and (2) multitask learning using distant supervision on their combination is an effective complete system on this challenging task.
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Question Answering | SQuAD1.1 dev | EM | 69.5 | DrQA (Document Reader only) |
| Question Answering | SQuAD1.1 dev | F1 | 78.8 | DrQA (Document Reader only) |
| Question Answering | Quasart-T | EM | 37.7 | DrQA |
| Question Answering | SQuAD1.1 | EM | 70.733 | Document Reader (single model) |
| Question Answering | SQuAD1.1 | F1 | 79.353 | Document Reader (single model) |
| Question Answering | Natural Questions (long) | F1 | 46.1 | DrQA |
| Question Answering | SearchQA | EM | 41.9 | DrQA |
| Question Answering | SQuAD1.1 | EM | 70 | DrQA |
| Open-Domain Question Answering | SearchQA | EM | 41.9 | DrQA |
| Open-Domain Question Answering | SQuAD1.1 | EM | 70 | DrQA |