Tasks SotA Datasets Papers Methods Submit About

Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable Benchmarks All SotA Datasets Papers Methods

Community

Submit Results About

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/Deepstruct zero-shot

Deepstruct zero-shot

Reported on 34 benchmarks across 5 tasks · 1 paper

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing26 results

Relation ExtractiononTACRED
F1· 2022-05-21
36.1
best: 86.6 (RAG4RE)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ExtractiononFewRel
F1 (10-way 1-shot)· 2022-05-21
67.6
best: 97.8 (DeepStruct multi-task w/ finetune)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ExtractiononFewRel
F1 (10-way 5-shot)· 2022-05-21
66.4
best: 99.8 (DeepStruct multi-task w/ finetune)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ExtractiononFewRel
F1 (5-way 1-shot)· 2022-05-21
72.4
best: 98.4 (DeepStruct multi-task w/ finetune)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ExtractiononFewRel
F1 (5-way 5-shot· 2022-05-21
70.8
best: 100 (DeepStruct multi-task w/ finetune)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ExtractiononACE2005
Entity F1· 2022-05-21
31.8
best: 90.2 (DeepStruct multi-task)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ExtractiononACE2005
Relation F1· 2022-05-21
5.3
best: 66.8 (DeepStruct multi-task w/ finetune)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ExtractiononNYT
Entity F1· 2022-05-21
60.5
best: 95.9 (DeepStruct multi-task w/ finetune)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ExtractiononNYT
Relation F1· 2022-05-21
28.6
best: 93.7 (DeepStruct multi-task)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ExtractiononCoNLL04
Entity F1· 2022-05-21
48.3
best: 90.7 (DeepStruct multi-task w/ finetune)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ExtractiononCoNLL04
Relation F1· 2022-05-21
25.8
best: 78.3 (DeepStruct multi-task w/ finetune)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ExtractiononADE Corpus
Entity F1· 2022-05-21
60.7
best: 91.1 (DeepStruct multi-task w/ finetune)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ExtractiononADE Corpus
Relation F1· 2022-05-21
10.6
best: 83.8 (DeepStruct multi-task w/ finetune)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ClassificationonTACRED
F1· 2022-05-21
36.1
best: 76.8 (DeepStruct multi-task w/ finetune)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ClassificationonFewRel
F1 (10-way 1-shot)· 2022-05-21
67.6
best: 97.8 (DeepStruct multi-task w/ finetune)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ClassificationonFewRel
F1 (10-way 5-shot)· 2022-05-21
66.4
best: 99.8 (DeepStruct multi-task w/ finetune)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ClassificationonFewRel
F1 (5-way 1-shot)· 2022-05-21
72.4
best: 98.4 (DeepStruct multi-task w/ finetune)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Relation ClassificationonFewRel
F1 (5-way 5-shot· 2022-05-21
70.8
best: 100 (DeepStruct multi-task w/ finetune)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Open Information ExtractiononWeb
F1· 2022-05-21
43.8
best: 91.2 (DeepEx (zero-shot))
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Open Information ExtractiononPenn Treebank
F1· 2022-05-21
51
best: 451 (DeepStruct multi-task w/ finetune)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Open Information ExtractiononNYT
F1· 2022-05-21
28.9
best: 85.5 (DeepEx (zero-shot))
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Open Information ExtractiononOIE2016
F1· 2022-05-21
28.1
best: 72.6 (DeepEx (zero-shot))
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Named Entity Recognition (NER)onCoNLL03
F1· 2022-05-21
44.4
best: 93.3 (UniNER-7B)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Named Entity Recognition (NER)onACE2005
F1· 2022-05-21
28.1
best: 86.9 (DeepStruct multi-task w/ finetune)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Named Entity Recognition (NER)onGENIA
F1· 2022-05-21
47.2
best: 81.77 (PIQN)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Named Entity Recognition (NER)onOntoNotes
F1· 2022-05-21
2.5
best: 87.8 (DeepStruct multi-task w/ finetune)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475

Medical8 results

Information ExtractiononACE2005
Entity F1· 2022-05-21
31.8
best: 90.2 (DeepStruct multi-task)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Information ExtractiononACE2005
Relation F1· 2022-05-21
5.3
best: 66.8 (DeepStruct multi-task w/ finetune)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Information ExtractiononNYT
Entity F1· 2022-05-21
60.5
best: 95.9 (DeepStruct multi-task w/ finetune)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Information ExtractiononNYT
Relation F1· 2022-05-21
28.6
best: 93.7 (DeepStruct multi-task)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Information ExtractiononCoNLL04
Entity F1· 2022-05-21
48.3
best: 90.7 (DeepStruct multi-task w/ finetune)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Information ExtractiononCoNLL04
Relation F1· 2022-05-21
25.8
best: 78.3 (DeepStruct multi-task w/ finetune)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Information ExtractiononADE Corpus
Entity F1· 2022-05-21
60.7
best: 91.1 (DeepStruct multi-task w/ finetune)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475
Information ExtractiononADE Corpus
Relation F1· 2022-05-21
10.6
best: 83.8 (DeepStruct multi-task w/ finetune)
DeepStruct: Pretraining of Language Models for Structure Prediction arXiv:2205.10475