TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

SotA/Natural Language Processing/Reading Comprehension

Reading Comprehension

28 benchmarks1760 papers

Most current question answering datasets frame the task as reading comprehension where the question is about a paragraph or document and the answer often is a span in the document.

Some specific tasks of reading comprehension include multi-modal machine reading comprehension and textual machine reading comprehension, among others. In the literature, machine reading comprehension can be divide into four categories: cloze style, multiple choice, span prediction, and free-form answer. Read more about each category here.

Benchmark datasets used for testing a model's reading comprehension abilities include MovieQA, ReCoRD, and RACE, among others.

The Machine Reading group at UCL also provides an overview of reading comprehension tasks.

Figure source: A Survey on Machine Reading Comprehension: Tasks, Evaluation Metrics and Benchmark Datasets

Benchmarks

Reading Comprehension on ReClor

TestAccuracyAccuracy (easy)Accuracy (hard)

Reading Comprehension on MuSeRC

Average F1EM

Reading Comprehension on BIG-bench

Accuracy Accuracy

Reading Comprehension on RACE

Accuracy (Middle)Accuracy (High)Accuracy

Reading Comprehension on MIntRec

Accuracy (20 classes)Accuracy (Binary)

Reading Comprehension on PhotoChat

F1PrecisionRecall

Reading Comprehension on MMDialog

F1

Reading Comprehension on AdversarialQA

Overall: F1D(BiDAF): F1D(BERT): F1D(RoBERTa): F1

Reading Comprehension on DREAM

Accuracy

Reading Comprehension on BIOMRC

Acc

Reading Comprehension on CrowdSource QA

MSE

Reading Comprehension on RadQA

Answer F1

Reading Comprehension on UQuAD

Exact MatchF1

Reading Comprehension on ReCAM

Accuracy