MoCo + wav2vec (w/o extLM)

Reported on 5 benchmarks across 5 tasks · 1 paper · 3 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Speech2 results

Audio-Visual Speech RecognitiononLRS2
Test WER· 2022-02-24
2.6
best: 1.4 (Whisper-Flamingo)
SOTA
Leveraging Unimodal Self-Supervised Learning for Multimodal Audio-Visual Speech Recognition arXiv:2203.07996
Automatic Speech Recognition (ASR)onLRS2
Test WER· 2022-02-24
2.7
best: 1.3 (Whisper)
SOTA
Leveraging Unimodal Self-Supervised Learning for Multimodal Audio-Visual Speech Recognition arXiv:2203.07996

Audio1 result

Speech RecognitiononLRS2
Test WER· 2022-02-24
2.7
best: 1.3 (Whisper)
SOTA
Leveraging Unimodal Self-Supervised Learning for Multimodal Audio-Visual Speech Recognition arXiv:2203.07996

Computer Vision1 result

LipreadingonLRS2
Word Error Rate (WER)· 2022-02-24
43.2
best: 14.6 (Auto-AVSR)
Leveraging Unimodal Self-Supervised Learning for Multimodal Audio-Visual Speech Recognition arXiv:2203.07996

Natural Language Processing1 result

Natural Language TransductiononLRS2
Word Error Rate (WER)· 2022-02-24
43.2
best: 14.6 (Auto-AVSR)
Leveraging Unimodal Self-Supervised Learning for Multimodal Audio-Visual Speech Recognition arXiv:2203.07996