Tasks SotA Datasets Papers Methods Submit About

Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable Benchmarks All SotA Datasets Papers Methods

Community

Submit Results About

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/Wav2Lip + GAN

Wav2Lip + GAN

Reported on 72 benchmarks across 9 tasks · 1 paper · 54 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Computer Vision32 results

Talking Head GenerationonLRS2
FID· uses extra data· 2020-08-23
4.446
best: 3.452 (Wav2Lip + ViT + MARLIN)
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Talking Head GenerationonLRS2
LSE-D· uses extra data· 2020-08-23
6.469
best: 7.127 (Wav2Lip + ViT + MARLIN)
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Talking Head GenerationonLRS3
FID· uses extra data· 2020-08-23
4.35
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Talking Head GenerationonLRS3
LSE-D· uses extra data· 2020-08-23
6.986
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Talking Head GenerationonLRW
FID· 2020-08-23
2.475
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Talking Head GenerationonLRW
LSE-D· 2020-08-23
6.774
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Face GenerationonLRS2
FID· uses extra data· 2020-08-23
4.446
best: 3.452 (Wav2Lip + ViT + MARLIN)
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Face GenerationonLRS2
LSE-D· uses extra data· 2020-08-23
6.469
best: 7.127 (Wav2Lip + ViT + MARLIN)
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Face GenerationonLRS3
FID· uses extra data· 2020-08-23
4.35
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Face GenerationonLRS3
LSE-D· uses extra data· 2020-08-23
6.986
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Face GenerationonLRW
FID· 2020-08-23
2.475
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Face GenerationonLRW
LSE-D· 2020-08-23
6.774
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Face ReconstructiononLRS2
FID· uses extra data· 2020-08-23
4.446
best: 3.452 (Wav2Lip + ViT + MARLIN)
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Face ReconstructiononLRS2
LSE-D· uses extra data· 2020-08-23
6.469
best: 7.127 (Wav2Lip + ViT + MARLIN)
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Face ReconstructiononLRS3
FID· uses extra data· 2020-08-23
4.35
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Face ReconstructiononLRS3
LSE-D· uses extra data· 2020-08-23
6.986
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Face ReconstructiononLRW
FID· 2020-08-23
2.475
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Face ReconstructiononLRW
LSE-D· 2020-08-23
6.774
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
3D Face ReconstructiononLRS2
FID· uses extra data· 2020-08-23
4.446
best: 3.452 (Wav2Lip + ViT + MARLIN)
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
3D Face ReconstructiononLRS2
LSE-D· uses extra data· 2020-08-23
6.469
best: 7.127 (Wav2Lip + ViT + MARLIN)
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
3D Face ReconstructiononLRS3
FID· uses extra data· 2020-08-23
4.35
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
3D Face ReconstructiononLRS3
LSE-D· uses extra data· 2020-08-23
6.986
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
3D Face ReconstructiononLRW
FID· 2020-08-23
2.475
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
3D Face ReconstructiononLRW
LSE-D· 2020-08-23
6.774
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Talking Head GenerationonLRS3
LSE-C· uses extra data· 2020-08-23
7.574
best: 7.887 (Wav2Lip)
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Talking Head GenerationonLRW
LSE-C· 2020-08-23
7.263
best: 7.49 (Wav2Lip)
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Face GenerationonLRS3
LSE-C· uses extra data· 2020-08-23
7.574
best: 7.887 (Wav2Lip)
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Face GenerationonLRW
LSE-C· 2020-08-23
7.263
best: 7.49 (Wav2Lip)
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Face ReconstructiononLRS3
LSE-C· uses extra data· 2020-08-23
7.574
best: 7.887 (Wav2Lip)
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Face ReconstructiononLRW
LSE-C· 2020-08-23
7.263
best: 7.49 (Wav2Lip)
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
3D Face ReconstructiononLRS3
LSE-C· uses extra data· 2020-08-23
7.574
best: 7.887 (Wav2Lip)
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
3D Face ReconstructiononLRW
LSE-C· 2020-08-23
7.263
best: 7.49 (Wav2Lip)
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010

Medical16 results

Image GenerationonLRS2
FID· uses extra data· 2020-08-23
4.446
best: 3.452 (Wav2Lip + ViT + MARLIN)
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Image GenerationonLRS2
LSE-D· uses extra data· 2020-08-23
6.469
best: 7.127 (Wav2Lip + ViT + MARLIN)
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Image GenerationonLRS3
FID· uses extra data· 2020-08-23
4.35
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Image GenerationonLRS3
LSE-D· uses extra data· 2020-08-23
6.986
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Image GenerationonLRW
FID· 2020-08-23
2.475
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Image GenerationonLRW
LSE-D· 2020-08-23
6.774
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
3D Face ModellingonLRS2
FID· uses extra data· 2020-08-23
4.446
best: 3.452 (Wav2Lip + ViT + MARLIN)
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
3D Face ModellingonLRS2
LSE-D· uses extra data· 2020-08-23
6.469
best: 7.127 (Wav2Lip + ViT + MARLIN)
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
3D Face ModellingonLRS3
FID· uses extra data· 2020-08-23
4.35
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
3D Face ModellingonLRS3
LSE-D· uses extra data· 2020-08-23
6.986
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
3D Face ModellingonLRW
FID· 2020-08-23
2.475
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
3D Face ModellingonLRW
LSE-D· 2020-08-23
6.774
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Image GenerationonLRS3
LSE-C· uses extra data· 2020-08-23
7.574
best: 7.887 (Wav2Lip)
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Image GenerationonLRW
LSE-C· 2020-08-23
7.263
best: 7.49 (Wav2Lip)
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
3D Face ModellingonLRS3
LSE-C· uses extra data· 2020-08-23
7.574
best: 7.887 (Wav2Lip)
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
3D Face ModellingonLRW
LSE-C· 2020-08-23
7.263
best: 7.49 (Wav2Lip)
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010

Music8 results

Facial Recognition and ModellingonLRS2
FID· uses extra data· 2020-08-23
4.446
best: 3.452 (Wav2Lip + ViT + MARLIN)
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Facial Recognition and ModellingonLRS2
LSE-D· uses extra data· 2020-08-23
6.469
best: 7.127 (Wav2Lip + ViT + MARLIN)
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Facial Recognition and ModellingonLRS3
FID· uses extra data· 2020-08-23
4.35
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Facial Recognition and ModellingonLRS3
LSE-D· uses extra data· 2020-08-23
6.986
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Facial Recognition and ModellingonLRW
FID· 2020-08-23
2.475
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Facial Recognition and ModellingonLRW
LSE-D· 2020-08-23
6.774
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Facial Recognition and ModellingonLRS3
LSE-C· uses extra data· 2020-08-23
7.574
best: 7.887 (Wav2Lip)
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
Facial Recognition and ModellingonLRW
LSE-C· 2020-08-23
7.263
best: 7.49 (Wav2Lip)
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010

Methodology8 results

3DonLRS2
FID· uses extra data· 2020-08-23
4.446
best: 3.452 (Wav2Lip + ViT + MARLIN)
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
3DonLRS2
LSE-D· uses extra data· 2020-08-23
6.469
best: 7.127 (Wav2Lip + ViT + MARLIN)
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
3DonLRS3
FID· uses extra data· 2020-08-23
4.35
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
3DonLRS3
LSE-D· uses extra data· 2020-08-23
6.986
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
3DonLRW
FID· 2020-08-23
2.475
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
3DonLRW
LSE-D· 2020-08-23
6.774
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
3DonLRS3
LSE-C· uses extra data· 2020-08-23
7.574
best: 7.887 (Wav2Lip)
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
3DonLRW
LSE-C· 2020-08-23
7.263
best: 7.49 (Wav2Lip)
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010

Audio8 results

10-shot image generationonLRS2
FID· uses extra data· 2020-08-23
4.446
best: 3.452 (Wav2Lip + ViT + MARLIN)
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
10-shot image generationonLRS2
LSE-D· uses extra data· 2020-08-23
6.469
best: 7.127 (Wav2Lip + ViT + MARLIN)
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
10-shot image generationonLRS3
FID· uses extra data· 2020-08-23
4.35
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
10-shot image generationonLRS3
LSE-D· uses extra data· 2020-08-23
6.986
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
10-shot image generationonLRW
FID· 2020-08-23
2.475
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
10-shot image generationonLRW
LSE-D· 2020-08-23
6.774
SOTA
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
10-shot image generationonLRS3
LSE-C· uses extra data· 2020-08-23
7.574
best: 7.887 (Wav2Lip)
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010
10-shot image generationonLRW
LSE-C· 2020-08-23
7.263
best: 7.49 (Wav2Lip)
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild arXiv:2008.10010