MQ-UpDown-C

Reported on 24 benchmarks across 1 task

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing24 results

Image Captioningonnocaps near-domain
B1
77.76
best: 88.9 (GIT2, Single Model)
Image Captioningonnocaps near-domain
B2
59
best: 75.86 (GIT2, Single Model)
Image Captioningonnocaps near-domain
B3
38.29
best: 58.99 (PaLI)
Image Captioningonnocaps near-domain
B4
21
best: 39.98 (PaLI)
Image Captioningonnocaps near-domain
CIDEr
76.34
best: 125.51 (GIT2, Single Model)
Image Captioningonnocaps near-domain
METEOR
25.59
best: 33.47 (PaLI)
Image Captioningonnocaps near-domain
ROUGE-L
53.15
best: 63.99 (PaLI)
Image Captioningonnocaps near-domain
SPICE
11.87
best: 16.11 (GIT2, Single Model)
Image Captioningonnocaps entire
B1
76.89
best: 88.1 (GIT, Single Model)
Image Captioningonnocaps entire
B2
57.76
best: 74.81 (GIT, Single Model)
Image Captioningonnocaps entire
B3
36.93
best: 57.68 (GIT, Single Model)
Image Captioningonnocaps entire
B4
20.11
best: 37.71 (CoCa - Google Brain)
Image Captioningonnocaps entire
CIDEr
75.58
best: 126.8 (Lyrics)
Image Captioningonnocaps entire
METEOR
25.18
best: 32.5 (GIT, Single Model)
Image Captioningonnocaps entire
ROUGE-L
52.53
best: 63.12 (GIT, Single Model)
Image Captioningonnocaps entire
SPICE
11.68
best: 15.94 (GIT, Single Model)
Image Captioningonnocaps in-domain
B1
78.73
best: 88.86 (GIT2, Single Model)
Image Captioningonnocaps in-domain
B2
61.63
best: 76.1 (GIT, Single Model)
Image Captioningonnocaps in-domain
B3
42.35
best: 60.53 (GIT, Single Model)
Image Captioningonnocaps in-domain
B4
25.94
best: 41.65 (GIT, Single Model)
Image Captioningonnocaps in-domain
CIDEr
80.19
best: 149.1 (PaLI)
Image Captioningonnocaps in-domain
METEOR
27.25
best: 34.22 (PaLI)
Image Captioningonnocaps in-domain
ROUGE-L
55.25
best: 64.39 (PaLI)
Image Captioningonnocaps in-domain
SPICE
12.38
best: 16.36 (GIT2, Single Model)