TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Papers/Self-Supervised Video Forensics by Audio-Visual Anomaly De...

Self-Supervised Video Forensics by Audio-Visual Anomaly Detection

Chao Feng, Ziyang Chen, Andrew Owens

2023-01-04CVPR 2023 1DeepFake DetectionAnomaly DetectionVideo Forensics
PaperPDFCode(official)

Abstract

Manipulated videos often contain subtle inconsistencies between their visual and audio signals. We propose a video forensics method, based on anomaly detection, that can identify these inconsistencies, and that can be trained solely using real, unlabeled data. We train an autoregressive model to generate sequences of audio-visual features, using feature sets that capture the temporal synchronization between video frames and sound. At test time, we then flag videos that the model assigns low probability. Despite being trained entirely on real videos, our model obtains strong performance on the task of detecting manipulated speech videos. Project site: https://cfeng16.github.io/audio-visual-forensics

Results

TaskDatasetMetricValueModel
3D ReconstructionFakeAVCelebAP94.2AVAD
3D ReconstructionFakeAVCelebROC AUC94.5AVAD
3DFakeAVCelebAP94.2AVAD
3DFakeAVCelebROC AUC94.5AVAD
DeepFake DetectionFakeAVCelebAP94.2AVAD
DeepFake DetectionFakeAVCelebROC AUC94.5AVAD
3D Shape Reconstruction from VideosFakeAVCelebAP94.2AVAD
3D Shape Reconstruction from VideosFakeAVCelebROC AUC94.5AVAD

Related Papers

Multi-Stage Prompt Inference Attacks on Enterprise LLM Systems2025-07-21SHIELD: A Secure and Highly Enhanced Integrated Learning for Robust Deepfake Detection against Adversarial Attacks2025-07-173DKeyAD: High-Resolution 3D Point Cloud Anomaly Detection via Keypoint-Guided Point Clustering2025-07-17A Semi-Supervised Learning Method for the Identification of Bad Exposures in Large Imaging Surveys2025-07-17A Privacy-Preserving Framework for Advertising Personalization Incorporating Federated Learning and Differential Privacy2025-07-16Bridge Feature Matching and Cross-Modal Alignment with Mutual-filtering for Zero-shot Anomaly Detection2025-07-15Adversarial Activation Patching: A Framework for Detecting and Mitigating Emergent Deception in Safety-Aligned Transformers2025-07-12Towards High-Resolution 3D Anomaly Detection: A Scalable Dataset and Real-Time Framework for Subtle Industrial Defects2025-07-10