Dataset for: "It is just a flu: Assessing the Effect of Watch History on YouTube's Pseudoscientific Video Recommendations"

Introduced 2020-10-22

The dataset consists of three files: the metadata, comments, and captions of the ground-truth dataset videos collected and manually reviewed in this paper.

  1. Video Metadata:
  • "groundtruth_videos.json": Contains the metadata of our manually reviewed ground-truth dataset videos. The ground-truth dataset includes 1,197 science, 1,325 pseudoscience, and 3,212 irrelevant videos. More specifically, it includes the metadata of videos related to the following pseudoscientific topics:
    • COVID-19: (607 science, 368 pseudoscience, 721 irrelevant videos)
    • Anti-vaccination (363 science, 394 pseudoscience, and 1,060 irrelevant videos)
    • Anti-mask (65 science, 188 pseudoscience, and 724 irrelevant videos)
    • Flat Earth (162 science, 375 pseudoscience, and 707 irrelevant videos)

Note, that 600 of the videos in this dataset include the "annotation.manual_review_label" attribute, which is the label assigned by the first author of this paper to evaluate the performance of the crowdsourced annotation process.

  • Video Metadata Description:
    • "search_term": The search terms used to search YouTube and retrieve these videos during our data collection. It can be one of the following search terms: 'covid-19', 'coronavirus', 'anti-vaccination', 'anti-vaxx', 'anti-mask', or 'flat earth'.
    • "annotation.annotations": The list of the three annotations assigned to each video by our crowdsourced annotators.
    • "annotation.label": The annotation label assigned to the video based on the majority agreement of the crowdsourced annotators.
    • "annotation.manual_review_label": The label assigned by the first author of this paper to evaluate the performance of the crowdsourced annotation process.
    • "isSeed": 0 if the video is a seed video of our data collection, 1 if it is a recommended video of a seed video.
    • "relatedVideos": The recommended videos of the given video as returned by the YouTube Data API.
  1. Video Comments:
  • "groundtruth_videos_comments_ids.json": Includes the identifiers of the comments of our ground-truth videos.
  1. Video Transcripts:
  • "groundtruth_videos_transcripts.json": Includes the captions of our ground-truth videos. If you use this dataset in any publication, of any form and kind, please cite using this data.