WiC-TSV
Words-in-Context: Target Sense Verification
TextsMITIntroduced 2020-04-30
WiC-TSV is a new multi-domain evaluation benchmark for Word Sense Disambiguation. More specifically, it is a framework for Target Sense Verification of Words in Context which grounds its uniqueness in the formulation as a binary classification task thus being independent of external sense inventories, and the coverage of various domains. This makes the dataset highly flexible for the evaluation of a diverse set of models and systems in and across domains. WiC-TSV provides three different evaluation settings, depending on the input signals provided to the model.
Benchmarks
Entity Linking/Task 1 Accuracy: allEntity Linking/Task 1 Accuracy: general purposeEntity Linking/Task 1 Accuracy: domain specificEntity Linking/Task 2 Accuracy: allEntity Linking/Task 2 Accuracy: general purposeEntity Linking/Task 2 Accuracy: domain specificEntity Linking/Task 3 Accuracy: allEntity Linking/Task 3 Accuracy: general purposeEntity Linking/Task 3 Accuracy: domain specificWord Sense Disambiguation/Task 1 Accuracy: allWord Sense Disambiguation/Task 1 Accuracy: general purposeWord Sense Disambiguation/Task 1 Accuracy: domain specificWord Sense Disambiguation/Task 2 Accuracy: allWord Sense Disambiguation/Task 2 Accuracy: general purposeWord Sense Disambiguation/Task 2 Accuracy: domain specificWord Sense Disambiguation/Task 3 Accuracy: allWord Sense Disambiguation/Task 3 Accuracy: general purposeWord Sense Disambiguation/Task 3 Accuracy: domain specific