Tasks SotA Datasets Papers Methods Submit About

Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable Benchmarks All SotA Datasets Papers Methods

Community

Submit Results About

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

KILT

KILT Benchmark

TextsIntroduced 2020-09-04

KILT (Knowledge Intensive Language Tasks) is a benchmark consisting of 11 datasets representing 5 types of tasks:

Fact-checking (FEVER),
Entity linking (AIDA CoNLL-YAGO, WNED-WIKI, WNED-CWEB),
Slot filling (T-Rex, Zero Shot RE),
Open domain QA (Natural Questions, HotpotQA, TriviaQA, ELI5),
Dialog generation (Wizard of Wikipedia).

All these datasets have been grounded in a single pre-processed wikipedia snapshot, allowing for fairer and more consistent evaluation as well as enabling new task setups such as multitask and transfer learning.

Source: KILT Benchmarking

Related Benchmarks

KILT: AIDA-YAGO2/Entity Linking/Accuracy KILT: AIDA-YAGO2/Entity Linking/KILT-AC KILT: AIDA-YAGO2/Entity Linking/R-Prec KILT: AIDA-YAGO2/Entity Linking/Recall@5 KILT: ELI5/Open-Domain Question Answering/F1 KILT: ELI5/Open-Domain Question Answering/KILT-F1 KILT: ELI5/Open-Domain Question Answering/KILT-RL KILT: ELI5/Open-Domain Question Answering/R-Prec KILT: ELI5/Open-Domain Question Answering/ROUGE-L KILT: ELI5/Open-Domain Question Answering/Recall@5 KILT: ELI5/Question Answering/F1 KILT: ELI5/Question Answering/KILT-F1 KILT: ELI5/Question Answering/KILT-RL KILT: ELI5/Question Answering/R-Prec KILT: ELI5/Question Answering/ROUGE-L KILT: ELI5/Question Answering/Recall@5 KILT: ELI5/Question Answering/Rouge-L KILT: FEVER/Fact Verification/Accuracy KILT: FEVER/Fact Verification/KILT-AC KILT: FEVER/Fact Verification/R-Prec KILT: FEVER/Fact Verification/Recall@5 KILT: HotpotQA/Open-Domain Question Answering/EM KILT: HotpotQA/Open-Domain Question Answering/F1 KILT: HotpotQA/Open-Domain Question Answering/KILT-EM KILT: HotpotQA/Open-Domain Question Answering/KILT-F1 KILT: HotpotQA/Open-Domain Question Answering/R-Prec KILT: HotpotQA/Open-Domain Question Answering/Recall@5 KILT: HotpotQA/Question Answering/EM KILT: HotpotQA/Question Answering/F1 KILT: HotpotQA/Question Answering/KILT-EM KILT: HotpotQA/Question Answering/KILT-F1 KILT: HotpotQA/Question Answering/R-Prec KILT: HotpotQA/Question Answering/Recall@5 KILT: Natural Questions/Open-Domain Question Answering/EM KILT: Natural Questions/Open-Domain Question Answering/F1 KILT: Natural Questions/Open-Domain Question Answering/KILT-EM KILT: Natural Questions/Open-Domain Question Answering/KILT-F1 KILT: Natural Questions/Open-Domain Question Answering/R-Prec KILT: Natural Questions/Open-Domain Question Answering/Recall@5 KILT: Natural Questions/Question Answering/EM KILT: Natural Questions/Question Answering/F1 KILT: Natural Questions/Question Answering/KILT-EM KILT: Natural Questions/Question Answering/KILT-F1 KILT: Natural Questions/Question Answering/R-Prec KILT: Natural Questions/Question Answering/Recall@5 KILT: T-REx/Slot Filling/Accuracy KILT: T-REx/Slot Filling/F1 KILT: T-REx/Slot Filling/KILT-AC KILT: T-REx/Slot Filling/KILT-F1 KILT: T-REx/Slot Filling/R-Prec KILT: T-REx/Slot Filling/Recall@5 KILT: TriviaQA/Open-Domain Question Answering/EM KILT: TriviaQA/Open-Domain Question Answering/F1 KILT: TriviaQA/Open-Domain Question Answering/KILT-EM KILT: TriviaQA/Open-Domain Question Answering/KILT-F1 KILT: TriviaQA/Open-Domain Question Answering/R-Prec KILT: TriviaQA/Open-Domain Question Answering/Recall@5 KILT: TriviaQA/Question Answering/EM KILT: TriviaQA/Question Answering/F1 KILT: TriviaQA/Question Answering/KILT-EM KILT: TriviaQA/Question Answering/KILT-F1 KILT: TriviaQA/Question Answering/R-Prec KILT: TriviaQA/Question Answering/Recall@5 KILT: WNED-CWEB/Entity Linking/Accuracy KILT: WNED-CWEB/Entity Linking/KILT-AC KILT: WNED-CWEB/Entity Linking/R-Prec KILT: WNED-CWEB/Entity Linking/Recall@5 KILT: WNED-WIKI/Entity Linking/Accuracy KILT: WNED-WIKI/Entity Linking/KILT-AC KILT: WNED-WIKI/Entity Linking/R-Prec KILT: WNED-WIKI/Entity Linking/Recall@5 KILT: Wizard of Wikipedia/Open-Domain Dialog/F1 KILT: Wizard of Wikipedia/Open-Domain Dialog/KILT-F1 KILT: Wizard of Wikipedia/Open-Domain Dialog/KILT-RL KILT: Wizard of Wikipedia/Open-Domain Dialog/R-Prec KILT: Wizard of Wikipedia/Open-Domain Dialog/ROUGE-L KILT: Wizard of Wikipedia/Open-Domain Dialog/Recall@5 KILT: Zero Shot RE/Slot Filling/Accuracy KILT: Zero Shot RE/Slot Filling/F1 KILT: Zero Shot RE/Slot Filling/KILT-AC KILT: Zero Shot RE/Slot Filling/KILT-F1 KILT: Zero Shot RE/Slot Filling/R-Prec KILT: Zero Shot RE/Slot Filling/Recall@5

Statistics

Papers: 117
Benchmarks: 0

Links

Tasks

Entity Linking Fact Verification Open-Domain Dialog Open-Domain Question Answering Question Answering Slot Filling