Tasks SotA Datasets Papers Methods Submit About

Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable Benchmarks All SotA Datasets Papers Methods

Community

Submit Results About

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/GPT2

GPT2

Reported on 25 benchmarks across 3 tasks · 2 papers · 3 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Methodology12 results

Multimodal Deep LearningonVALSE actant swap
pairwise accuracy· 2021-12-14
76.9
SOTA
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena arXiv:2112.07566
Multimodal Deep LearningonVALSE foil-it (noun phrases)
pairwise accuracy· 2021-12-14
80.7
best: 88.8 (CLIP)
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena arXiv:2112.07566
Multimodal Deep LearningonVALSE counting adversarial
pairwise accuracy· 2021-12-14
45.3
best: 77.3 (ViLBERT 12-in-1)
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena arXiv:2112.07566
Multimodal Deep LearningonVALSE counting balanced
pairwise accuracy· 2021-12-14
51.6
best: 76.7 (ViLBERT 12-in-1)
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena arXiv:2112.07566
Multimodal Deep LearningonVALSE coreference clean
pairwise accuracy· 2021-12-14
50
best: 69.2 (ViLBERT 12-in-1)
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena arXiv:2112.07566
Multimodal Deep LearningonVALSE counting small numbers
pairwise accuracy· 2021-12-14
49.8
best: 80.2 (ViLBERT 12-in-1)
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena arXiv:2112.07566
Multimodal Deep LearningonVALSE existence
pairwise accuracy· 2021-12-14
58
best: 95.6 (ViLBERT 12-in-1)
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena arXiv:2112.07566
Multimodal Deep LearningonVALSE coreference standard
pairwise accuracy· 2021-12-14
54.5
best: 75.7 (ViLBERT 12-in-1)
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena arXiv:2112.07566
Multimodal Deep LearningonVALSE spatial relations
pairwise accuracy· 2021-12-14
75
best: 77.2 (GPT1)
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena arXiv:2112.07566
Multimodal Deep LearningonVALSE plurality
pairwise accuracy· 2021-12-14
51.9
best: 72.4 (ViLBERT 12-in-1)
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena arXiv:2112.07566
Multimodal Deep LearningonVALSE action replacement
pairwise accuracy· 2021-12-14
66.8
best: 75.6 (CLIP)
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena arXiv:2112.07566
Multimodal Deep LearningonVALSE
average pairwise accuracy· 2021-12-14
60.1
best: 75.1 (ViLBERT 12-in-1)
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena arXiv:2112.07566

Natural Language Processing12 results

Multimodal Text and Image ClassificationonVALSE actant swap
pairwise accuracy· 2021-12-14
76.9
SOTA
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena arXiv:2112.07566
Multimodal Text and Image ClassificationonVALSE foil-it (noun phrases)
pairwise accuracy· 2021-12-14
80.7
best: 88.8 (CLIP)
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena arXiv:2112.07566
Multimodal Text and Image ClassificationonVALSE counting adversarial
pairwise accuracy· 2021-12-14
45.3
best: 77.3 (ViLBERT 12-in-1)
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena arXiv:2112.07566
Multimodal Text and Image ClassificationonVALSE counting balanced
pairwise accuracy· 2021-12-14
51.6
best: 76.7 (ViLBERT 12-in-1)
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena arXiv:2112.07566
Multimodal Text and Image ClassificationonVALSE coreference clean
pairwise accuracy· 2021-12-14
50
best: 69.2 (ViLBERT 12-in-1)
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena arXiv:2112.07566
Multimodal Text and Image ClassificationonVALSE counting small numbers
pairwise accuracy· 2021-12-14
49.8
best: 80.2 (ViLBERT 12-in-1)
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena arXiv:2112.07566
Multimodal Text and Image ClassificationonVALSE existence
pairwise accuracy· 2021-12-14
58
best: 95.6 (ViLBERT 12-in-1)
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena arXiv:2112.07566
Multimodal Text and Image ClassificationonVALSE coreference standard
pairwise accuracy· 2021-12-14
54.5
best: 75.7 (ViLBERT 12-in-1)
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena arXiv:2112.07566
Multimodal Text and Image ClassificationonVALSE spatial relations
pairwise accuracy· 2021-12-14
75
best: 77.2 (GPT1)
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena arXiv:2112.07566
Multimodal Text and Image ClassificationonVALSE plurality
pairwise accuracy· 2021-12-14
51.9
best: 72.4 (ViLBERT 12-in-1)
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena arXiv:2112.07566
Multimodal Text and Image ClassificationonVALSE action replacement
pairwise accuracy· 2021-12-14
66.8
best: 75.6 (CLIP)
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena arXiv:2112.07566
Multimodal Text and Image ClassificationonVALSE
average pairwise accuracy· 2021-12-14
60.1
best: 75.1 (ViLBERT 12-in-1)
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena arXiv:2112.07566

Medical1 result

Language Modellingonlanguage-modeling-recommendation
1:1 Accuracy· 2021-12-08
48.8
SOTA
Zero-Shot Recommendation as Language Modeling arXiv:2112.04184