Whisper-Flamingo

Reported on 2 benchmarks across 1 task · 1 paper · 2 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Speech2 results

Audio-Visual Speech RecognitiononLRS3-TED
Word Error Rate (WER)· uses extra data· 2024-06-14
0.76
best: 0.74 (MMS-LLaMA)
SOTA
Whisper-Flamingo: Integrating Visual Features into Whisper for Audio-Visual Speech Recognition and Translation arXiv:2406.10082
Audio-Visual Speech RecognitiononLRS2
Test WER· uses extra data· 2024-06-14
1.4
SOTA
Whisper-Flamingo: Integrating Visual Features into Whisper for Audio-Visual Speech Recognition and Translation arXiv:2406.10082