Rudolf Kadlec, Martin Schmid, Ondrej Bajgar, Jan Kleindienst
Several large cloze-style context-question-answer datasets have been introduced recently: the CNN and Daily Mail news data and the Children's Book Test. Thanks to the size of these datasets, the associated text comprehension task is well suited for deep-learning techniques that currently seem to outperform all alternative approaches. We present a new, simple model that uses attention to directly pick the answer from the context as opposed to computing the answer using a blended representation of words in the document as is usual in similar models. This makes the model particularly suitable for question-answering problems where the answer is a single word from the document. Ensemble of our models sets new state of the art on all evaluated datasets.
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Question Answering | CNN / Daily Mail | CNN | 75.4 | AS Reader (ensemble model) |
| Question Answering | CNN / Daily Mail | Daily Mail | 77.7 | AS Reader (ensemble model) |
| Question Answering | CNN / Daily Mail | CNN | 69.5 | AS Reader (single model) |
| Question Answering | CNN / Daily Mail | Daily Mail | 73.9 | AS Reader (single model) |
| Question Answering | SearchQA | N-gram F1 | 22.8 | ASR |
| Question Answering | SearchQA | Unigram Acc | 41.3 | ASR |
| Open-Domain Question Answering | SearchQA | N-gram F1 | 22.8 | ASR |
| Open-Domain Question Answering | SearchQA | Unigram Acc | 41.3 | ASR |