CREPE
TextsIntroduced 2022-11-30
CREPE is QA dataset containing a natural distribution of presupposition failures from online information-seeking forums. It consists of 8400 Reddit questions with (1) whether there is any false presuppositions annotated, and (2) if any, the presupposition and its correction written.
Source: CREPE: Open-Domain Question Answering with False Presuppositions
Image Source: https://arxiv.org/pdf/2211.17257v1.pdf
Related Benchmarks
CREPE (Compositional REPresentation Evaluation)/Image Retrieval/Recall@1 (HN-Atom + HN-Comp, SC)CREPE (Compositional REPresentation Evaluation)/Image Retrieval/Recall@1 (HN-Atom + HN-Comp, UC)CREPE (Compositional REPresentation Evaluation)/Image Retrieval/Recall@1 (HN-Atom, UC)CREPE (Compositional REPresentation Evaluation)/Image Retrieval/Recall@1 (HN-Comp, UC)