WikiEvalFacts

WikiEvalFacts This is an augmented version of the WikiEval dataset which additionally includes generated fact statements for each QA pair and human annotation of their truthfulness against each answer type.

This is the dataset used in our 2024 paper FaaF: Facts as a Function for the evaluation of generated text.

Column description question: a question that can be answered from the given Wikipedia page (source). source: The source Wikipedia page from which the question and context are generated. gt_facts: A set of ground truth fact statemets generated using the answer and question columns. human_evaluated_facts_answer: Human annotation of the truthfulness of each of the fact ground truth statements against the content in the answer column. human_evaluated_facts_ungrounded_answer: Human annotation of the truthfulness of each of the fact ground truth statements against the content in the ungrounded_answer column. human_evaluated_facts_poor_answer: Human annotation of the truthfulness of each of the fact ground truth statements against the content in the poor_answer column. unique_id: a simple integer index. answer: answer grounded on context_v1 ungrounded_answer: answer generated without context_v1 poor_answer: answer with poor relevancy compared to grounded_answer and ungrounded_answer context_v1: Ideal context to answer the given question contetx_v2: context that contains redundant information compared to context_v1