Barlas Oguz, Xilun Chen, Vladimir Karpukhin, Stan Peshterliev, Dmytro Okhonko, Michael Schlichtkrull, Sonal Gupta, Yashar Mehdad, Scott Yih
We study open-domain question answering with structured, unstructured and semi-structured knowledge sources, including text, tables, lists and knowledge bases. Departing from prior work, we propose a unifying approach that homogenizes all sources by reducing them to text and applies the retriever-reader model which has so far been limited to text sources only. Our approach greatly improves the results on knowledge-base QA tasks by 11 points, compared to latest graph-based methods. More importantly, we demonstrate that our unified knowledge (UniK-QA) model is a simple and yet effective way to combine heterogeneous sources of knowledge, advancing the state-of-the-art results on two popular question answering benchmarks, NaturalQuestions and WebQuestions, by 3.5 and 2.6 points, respectively. The code of UniK-QA is available at: https://github.com/facebookresearch/UniK-QA.
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Question Answering | TIQ | P@1 | 42.5 | Unik-Qa |
| Question Answering | Natural Questions (long) | EM | 54.9 | UniK-QA |
| Question Answering | WebQuestions | Exact Match | 57.7 | UniK-QA |
| Question Answering | Natural Questions | Exact Match | 54.9 | UniK-QA |
| Question Answering | TQA | Exact Match | 65.5 | UniK-QA |
| Question Answering | WebQuestionsSP | Hits@1 | 79.1 | UniK-QA (T5-large) |
| Question Answering | WebQuestionsSP | Hits@1 | 76.7 | UniK-QA (T5-base) |
| Open-Domain Question Answering | WebQuestions | Exact Match | 57.7 | UniK-QA |
| Open-Domain Question Answering | Natural Questions | Exact Match | 54.9 | UniK-QA |
| Open-Domain Question Answering | TQA | Exact Match | 65.5 | UniK-QA |