Image Captioning on COCO Captions

Metric: METEOR (higher is better)

LeaderboardDataset

Loading chart...

Results

Sort:

#	Model↕	METEOR▼	Extra Data	Paper	Date↕	Code
1	CoCa	33.9	No	CoCa: Contrastive Captioners are Image-Text Foun...	2022-05-04	Code
2	SimVLM	33.4	No	SimVLM: Simple Visual Language Model Pretraining...	2021-08-24	Code
3	OFA	32.5	No	OFA: Unifying Architectures, Tasks, and Modaliti...	2022-02-07	Code
4	GIT	32.2	No	GIT: A Generative Image-to-text Transformer for ...	2022-05-27	Code
5	mPLUG	32	No	mPLUG: Effective and Efficient Vision-Language L...	2022-05-24	Code
6	Prompt Tuning	31.51	No	Prompt Tuning for Generative Multimodal Pretrain...	2022-08-04	Code
7	LEMON	31.4	No	Scaling Up Vision-Language Pre-training for Imag...	2021-11-24	-
8	Prismer	31.4	No	Prismer: A Vision-Language Model with Multi-Task...	2023-03-04	Code
9	L-Verse	31.4	No	L-Verse: Bidirectional Generation Between Image ...	2021-11-22	Code
10	VinVL	31.1	No	VinVL: Revisiting Visual Representations in Visi...	2021-01-02	Code
11	ExpansionNet v2 (No VL pretraining)	30.6	No	Exploiting Multiple Sequence Lengths in Fast End...	2022-08-13	Code
12	GRIT (No VL pretraining - base)	30.6	No	GRIT: Faster and Better Image captioning Transfo...	2022-07-20	Code
13	Oscar	30.6	No	Oscar: Object-Semantics Aligned Pre-training for...	2020-04-13	Code
14	Xmodal-Ctx	30.4	No	Beyond a Pre-Trained Object Detector: Cross-Moda...	2022-05-09	Code
15	PTP-BLIP (14M)	30.4	No	Position-guided Text Prompt for Vision-Language ...	2022-12-19	Code
16	Xmodal-Ctx	30	No	Beyond a Pre-Trained Object Detector: Cross-Moda...	2022-05-09	Code
17	X-Transformer	29.5	No	X-Linear Attention Networks for Image Captioning	2020-03-31	Code
18	LaDiC (ours, 30 steps)	29.5	No	LaDiC: Are Diffusion Models Really Inferior to A...	2024-04-16	Code
19	AoANet + VC	29.3	No	Visual Commonsense R-CNN	2020-02-27	Code
20	Meshed-Memory Transformer	29.2	No	Meshed-Memory Transformer for Image Captioning	2019-12-17	Code
21	Transformer_NSC	28.9	No	A Better Variant of Self-Critical Sequence Train...	2020-03-22	Code
22	CLIP Text Encoder (RL w/ CIDEr-reward)	28.7	No	Fine-grained Image Captioning with CLIP Reward	2022-05-26	Code
23	RefineCap (w/ REINFORCE)	28.3	No	RefineCap: Concept-Aware Refinement for Image Ca...	2021-09-08	-
24	SmallCapd=16, Large	28.3	No	SmallCap: Lightweight Image Captioning Prompted ...	2022-09-30	Code
25	RDN	28.1	No	Reflective Decoding Network for Image Captioning	2019-08-30	-
26	ClipCap (Transformer)	27.45	No	ClipCap: CLIP Prefix for Image Captioning	2021-11-18	Code
27	ClipCap (MLP + GPT2 tuning)	27.1	No	ClipCap: CLIP Prefix for Image Captioning	2021-11-18	Code
28	CapDec	25.1	No	Text-Only Training for Image Captioning using No...	2022-11-01	Code
29	From Captions to Visual Concepts and Back	23.6	No	From Captions to Visual Concepts and Back	2014-11-18	Code
30	VLKD (ViT-B/16)	19.7	No	-	-	-

#1CoCaSOTA
33.9
METEOR· 2022-05-04
CoCa: Contrastive Captioners are Image-Text Foundation Models Code
#2SimVLMSOTA
33.4
METEOR· 2021-08-24
SimVLM: Simple Visual Language Model Pretraining with Weak Supervision Code
#3OFA
32.5
METEOR· 2022-02-07
OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework Code
#4GIT
32.2
METEOR· 2022-05-27
GIT: A Generative Image-to-text Transformer for Vision and Language Code
#5mPLUG
32
METEOR· 2022-05-24
mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections Code
#6Prompt Tuning
31.51
METEOR· 2022-08-04
Prompt Tuning for Generative Multimodal Pretrained Models Code
#7LEMON
31.4
METEOR· 2021-11-24
Scaling Up Vision-Language Pre-training for Image Captioning
#8Prismer
31.4
METEOR· 2023-03-04
Prismer: A Vision-Language Model with Multi-Task Experts Code
#9L-Verse
31.4
METEOR· 2021-11-22
L-Verse: Bidirectional Generation Between Image and Text Code
#10VinVLSOTA
31.1
METEOR· 2021-01-02
VinVL: Revisiting Visual Representations in Vision-Language Models Code
#11ExpansionNet v2 (No VL pretraining)
30.6
METEOR· 2022-08-13
Exploiting Multiple Sequence Lengths in Fast End to End Training for Image Captioning Code
#12GRIT (No VL pretraining - base)
30.6
METEOR· 2022-07-20
GRIT: Faster and Better Image captioning Transformer Using Dual Visual Features Code
#13OscarSOTA
30.6
METEOR· 2020-04-13
Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks Code
#14Xmodal-Ctx
30.4
METEOR· 2022-05-09
Beyond a Pre-Trained Object Detector: Cross-Modal Textual and Visual Context for Image Captioning Code
#15PTP-BLIP (14M)
30.4
METEOR· 2022-12-19
Position-guided Text Prompt for Vision-Language Pre-training Code
#16Xmodal-Ctx
30
METEOR· 2022-05-09
Beyond a Pre-Trained Object Detector: Cross-Modal Textual and Visual Context for Image Captioning Code
#17X-TransformerSOTA
29.5
METEOR· 2020-03-31
X-Linear Attention Networks for Image Captioning Code
#18LaDiC (ours, 30 steps)
29.5
METEOR· 2024-04-16
LaDiC: Are Diffusion Models Really Inferior to Autoregressive Counterparts for Image-to-Text Generation?Code
#19AoANet + VCSOTA
29.3
METEOR· 2020-02-27
Visual Commonsense R-CNN Code
#20Meshed-Memory TransformerSOTA
29.2
METEOR· 2019-12-17
Meshed-Memory Transformer for Image Captioning Code
#21Transformer_NSC
28.9
METEOR· 2020-03-22
A Better Variant of Self-Critical Sequence Training Code
#22CLIP Text Encoder (RL w/ CIDEr-reward)
28.7
METEOR· 2022-05-26
Fine-grained Image Captioning with CLIP Reward Code
#23RefineCap (w/ REINFORCE)
28.3
METEOR· 2021-09-08
RefineCap: Concept-Aware Refinement for Image Captioning
#24SmallCapd=16, Large
28.3
METEOR· 2022-09-30
SmallCap: Lightweight Image Captioning Prompted with Retrieval Augmentation Code
#25RDNSOTA
28.1
METEOR· 2019-08-30
Reflective Decoding Network for Image Captioning
#26ClipCap (Transformer)
27.45
METEOR· 2021-11-18
ClipCap: CLIP Prefix for Image Captioning Code
#27ClipCap (MLP + GPT2 tuning)
27.1
METEOR· 2021-11-18
ClipCap: CLIP Prefix for Image Captioning Code
#28CapDec
25.1
METEOR· 2022-11-01
Text-Only Training for Image Captioning using Noise-Injected CLIP Code
#29From Captions to Visual Concepts and BackSOTA
23.6
METEOR· 2014-11-18
From Captions to Visual Concepts and Back Code
#30VLKD (ViT-B/16)
19.7
METEOR
No paper