Audio captioning on AudioCaps

Metric: SPIDEr (higher is better)

LeaderboardDataset

Loading chart...

Results

Hide extra data

Sort:

#	Model↕	SPIDEr▼	Extra Data	Paper	Date↕	Code
1	Audio Flamingo	32.6	Yes	Audio Flamingo: A Novel Audio Language Model wit...	2024-02-02	Code
2	ZerAuCap	18.3	Yes	Zero-shot audio captioning with audio-language m...	2023-11-14	Code
3	MQ-Cap	0.519	Yes	Enhancing Retrieval-Augmented Audio Captioning w...	2024-10-14	-
4	SLAM-AAC	0.518	Yes	SLAM-AAC: Enhancing Audio Captioning with Paraph...	2024-10-12	Code
5	LAVCap	0.517	No	LAVCap: LLM-based Audio-Visual Captioning using ...	2025-01-16	Code
6	EnCLAP++-large	0.51	Yes	EnCLAP++: Analyzing the EnCLAP Framework for Opt...	2024-09-02	Code
7	AutoCap	0.507	No	Taming Data and Transformers for Audio Generation	2024-06-27	Code
8	LOAE	0.505	Yes	Enhancing Automated Audio Captioning via Large L...	2024-06-19	Code
9	EnCLAP++-base	0.501	Yes	EnCLAP++: Analyzing the EnCLAP Framework for Opt...	2024-09-02	Code
10	EnCLAP-large	0.4954	No	EnCLAP: Combining Neural Audio Codec and Audio-T...	2024-01-31	Code
11	CNext-trans	0.4951	No	-	-	-
12	EnCLAP-base	0.4829	No	EnCLAP: Combining Neural Audio Codec and Audio-T...	2024-01-31	Code
13	AL-MixGen + Multi-TTA	0.475	No	-	-	-
14	Rethink-ACT (AST + TF + MIL)	0.472	No	-	-	-
15	AL-MixGen	0.466	No	Exploring Train and Test-Time Augmentations for ...	2022-10-31	-
16	BART + YAMNet + PANNs	0.465	No	-	-	Code
17	CNN+Transformer	0.426	No	Audio Captioning Transformer	2021-07-21	Code
18	TopDown-AlignedAtt (1NN)	0.369	No	-	-	-
19	No audio (baseline)	0	No	-	-	Code

#1Audio FlamingoSOTA
32.6
SPIDEr· Extra Data· 2024-02-02
Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities Code
#2ZerAuCapSOTA
18.3
SPIDEr· Extra Data· 2023-11-14
Zero-shot audio captioning with audio-language model guidance and audio context keywords Code
#3MQ-Cap
0.519
SPIDEr· Extra Data· 2024-10-14
Enhancing Retrieval-Augmented Audio Captioning with Generation-Assisted Multimodal Querying and Progressive Learning
#4SLAM-AAC
0.518
SPIDEr· Extra Data· 2024-10-12
SLAM-AAC: Enhancing Audio Captioning with Paraphrasing Augmentation and CLAP-Refine through LLMs Code
#5LAVCap
0.517
SPIDEr· 2025-01-16
LAVCap: LLM-based Audio-Visual Captioning using Optimal Transport Code
#6EnCLAP++-large
0.51
SPIDEr· Extra Data· 2024-09-02
EnCLAP++: Analyzing the EnCLAP Framework for Optimizing Automated Audio Captioning Performance Code
#7AutoCap
0.507
SPIDEr· 2024-06-27
Taming Data and Transformers for Audio Generation Code
#8LOAE
0.505
SPIDEr· Extra Data· 2024-06-19
Enhancing Automated Audio Captioning via Large Language Models with Optimized Audio Encoding Code
#9EnCLAP++-base
0.501
SPIDEr· Extra Data· 2024-09-02
EnCLAP++: Analyzing the EnCLAP Framework for Optimizing Automated Audio Captioning Performance Code
#10EnCLAP-large
0.4954
SPIDEr· 2024-01-31
EnCLAP: Combining Neural Audio Codec and Audio-Text Joint Embedding for Automated Audio Captioning Code
#11CNext-trans
0.4951
SPIDEr
No paper
#12EnCLAP-base
0.4829
SPIDEr· 2024-01-31
EnCLAP: Combining Neural Audio Codec and Audio-Text Joint Embedding for Automated Audio Captioning Code
#13AL-MixGen + Multi-TTA
0.475
SPIDEr
No paper
#14Rethink-ACT (AST + TF + MIL)
0.472
SPIDEr
No paper
#15AL-MixGenSOTA
0.466
SPIDEr· 2022-10-31
Exploring Train and Test-Time Augmentations for Audio-Language Learning
#16BART + YAMNet + PANNs
0.465
SPIDEr
No paperCode
#17CNN+TransformerSOTA
0.426
SPIDEr· 2021-07-21
Audio Captioning Transformer Code
#18TopDown-AlignedAtt (1NN)
0.369
SPIDEr
No paper
#19No audio (baseline)
0
SPIDEr
No paperCode