Tasks SotA Datasets Papers Methods Submit About

Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable Benchmarks All SotA Datasets Papers Methods

Community

Submit Results About

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/DeepStruct multi-task w/ finetune

DeepStruct multi-task w/ finetune

Reported on 54 benchmarks across 9 tasks · 1 paper · 34 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing42 results

Relation ExtractiononTACRED
F1· 2022-05-21
76.8
best: 86.6 (RAG4RE)
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ExtractiononFewRel
F1 (10-way 1-shot)· 2022-05-21
97.8
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ExtractiononFewRel
F1 (10-way 5-shot)· 2022-05-21
99.8
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ExtractiononFewRel
F1 (5-way 1-shot)· 2022-05-21
98.4
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ExtractiononFewRel
F1 (5-way 5-shot· 2022-05-21
100
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ExtractiononACE2005
Relation F1· 2022-05-21
66.8
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ExtractiononNYT
Entity F1· 2022-05-21
95.9
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ExtractiononCoNLL04
Entity F1· 2022-05-21
90.7
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ExtractiononCoNLL04
Relation F1· 2022-05-21
78.3
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ExtractiononADE Corpus
Entity F1· 2022-05-21
91.1
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ExtractiononADE Corpus
Relation F1· 2022-05-21
83.8
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ClassificationonTACRED
F1· 2022-05-21
76.8
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ClassificationonFewRel
F1 (10-way 1-shot)· 2022-05-21
97.8
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ClassificationonFewRel
F1 (10-way 5-shot)· 2022-05-21
99.8
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ClassificationonFewRel
F1 (5-way 1-shot)· 2022-05-21
98.4
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ClassificationonFewRel
F1 (5-way 5-shot· 2022-05-21
100
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Open Information ExtractiononPenn Treebank
F1· 2022-05-21
451
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Open Information ExtractiononACE2005
Trigger Id· 2022-05-21
73.5
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Semantic Role LabelingonCoNLL05 Brown
F1· 2022-05-21
92.1
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Named Entity Recognition (NER)onACE2005
F1· 2022-05-21
86.9
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Named Entity Recognition (NER)onOntoNotes
F1· 2022-05-21
87.8
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Coreference ResolutiononCoNLL12
Average F1· 2022-05-21
73.1
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Coreference ResolutiononCoNLL12
B3· 2022-05-21
71.3
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Coreference ResolutiononCoNLL12
CEAFϕ4· 2022-05-21
73.1
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Coreference ResolutiononCoNLL12
MUC· 2022-05-21
74.9
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Event ExtractiononACE2005
Trigger Id· 2022-05-21
73.5
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ExtractiononTACRED
F1· 2022-05-21
76.8
best: 86.6 (RAG4RE)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ExtractiononACE2005
Entity F1· 2022-05-21
90
best: 90.2 (DeepStruct multi-task)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ExtractiononNYT
Relation F1· 2022-05-21
93.3
best: 93.7 (DeepStruct multi-task)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Open Information ExtractiononWeb
F1· 2022-05-21
49.1
best: 91.2 (DeepEx (zero-shot))
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Open Information ExtractiononNYT
F1· 2022-05-21
45
best: 85.5 (DeepEx (zero-shot))
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Open Information ExtractiononOIE2016
F1· 2022-05-21
71.3
best: 72.6 (DeepEx (zero-shot))
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Open Information ExtractiononACE2005
Argument Cl· 2022-05-21
56.2
best: 63.9 (DeepStruct multi-task)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Open Information ExtractiononACE2005
Argument Id· 2022-05-21
59.4
best: 67.5 (DeepStruct multi-task)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Open Information ExtractiononACE2005
Trigger Cl· 2022-05-21
69.8
best: 71.9 (Text2Event - T5-large)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Semantic Role LabelingonCoNLL05 WSJ
F1· 2022-05-21
95.2
best: 95.5 (DeepStruct multi-task)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Semantic Role LabelingonCoNLL12
F1· 2022-05-21
96
best: 97.2 (DeepStruct multi-task)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Named Entity Recognition (NER)onCoNLL03
F1· 2022-05-21
93
best: 93.3 (UniNER-7B)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Named Entity Recognition (NER)onGENIA
F1· 2022-05-21
80.8
best: 81.77 (PIQN)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Event ExtractiononACE2005
Argument Cl· 2022-05-21
56.2
best: 63.9 (DeepStruct multi-task)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Event ExtractiononACE2005
Argument Id· 2022-05-21
59.4
best: 67.5 (DeepStruct multi-task)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Event ExtractiononACE2005
Trigger Cl· 2022-05-21
69.8
best: 71.9 (Text2Event - T5-large)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475

Medical12 results

Information ExtractiononACE2005
Trigger Id· 2022-05-21
73.5
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Information ExtractiononACE2005
Relation F1· 2022-05-21
66.8
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Information ExtractiononNYT
Entity F1· 2022-05-21
95.9
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Information ExtractiononCoNLL04
Entity F1· 2022-05-21
90.7
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Information ExtractiononCoNLL04
Relation F1· 2022-05-21
78.3
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Information ExtractiononADE Corpus
Entity F1· 2022-05-21
91.1
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Information ExtractiononADE Corpus
Relation F1· 2022-05-21
83.8
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Information ExtractiononACE2005
Argument Cl· 2022-05-21
56.2
best: 63.9 (DeepStruct multi-task)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Information ExtractiononACE2005
Argument Id· 2022-05-21
59.4
best: 67.5 (DeepStruct multi-task)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Information ExtractiononACE2005
Trigger Cl· 2022-05-21
69.8
best: 71.9 (Text2Event - T5-large)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Information ExtractiononACE2005
Entity F1· 2022-05-21
90
best: 90.2 (DeepStruct multi-task)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Information ExtractiononNYT
Relation F1· 2022-05-21
93.3
best: 93.7 (DeepStruct multi-task)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475

Speech1 result

DialogueonMULTIWOZ 2.1
Joint Acc· 2022-05-21
54.2
SOTA
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475