3D Conv + ResNet-18 + MS-TCN + Multi-Head Visual-Audio Memory

Reported on 4 benchmarks across 2 tasks · 1 paper · 2 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Computer Vision2 results

LipreadingonCAS-VSR-W1k (LRW-1000)
Top-1 Accuracy· 2022-04-04
53.8
best: 58.2 (SyncVSR (Word Boundary))
SOTA
Distinguishing Homophenes Using Multi-Head Visual-Audio Memory for Lip Reading arXiv:2204.01725
LipreadingonLip Reading in the Wild
Top-1 Accuracy· 2022-04-04
88.5
best: 95 (SyncVSR (Word Boundary))
Distinguishing Homophenes Using Multi-Head Visual-Audio Memory for Lip Reading arXiv:2204.01725

Natural Language Processing2 results

Natural Language TransductiononCAS-VSR-W1k (LRW-1000)
Top-1 Accuracy· 2022-04-04
53.8
best: 58.2 (SyncVSR (Word Boundary))
SOTA
Distinguishing Homophenes Using Multi-Head Visual-Audio Memory for Lip Reading arXiv:2204.01725
Natural Language TransductiononLip Reading in the Wild
Top-1 Accuracy· 2022-04-04
88.5
best: 95 (SyncVSR (Word Boundary))
Distinguishing Homophenes Using Multi-Head Visual-Audio Memory for Lip Reading arXiv:2204.01725