BCOPA-CE

A Balanced COPA Test Set with cause-effect as alternatives

TextsIntroduced 2021-07-05

We provide the BCOPA-CE test set, which has balanced token distribution in the correct and wrong alternatives and increases the difficulty of being aware of cause and effect.

construction

  1. for each premise of the 500 samples in COPA-test set, we generate one event manually which is a plausible answer to the opposite question type of the original sample.
  2. obtain 500 triplets of <premise, cause, effect>
  3. construct 1000 samples by giving two different questions (cause or effect) to each triplet.