Enhance Multimodal Transformer With External Label And In-Domain Pretrain: Hateful Meme Challenge Winning Solution
Ron Zhu
2020-12-15Meme Classification
Abstract
Hateful meme detection is a new research area recently brought out that requires both visual, linguistic understanding of the meme and some background knowledge to performing well on the task. This technical report summarises the first place solution of the Hateful Meme Detection Challenge 2020, which extending state-of-the-art visual-linguistic transformers to tackle this problem. At the end of the report, we also point out the shortcomings and possible directions for improving the current methodology.
Results
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Meme Classification | Hateful Memes | Accuracy | 0.732 | Ron Zhu |
| Meme Classification | Hateful Memes | ROC-AUC | 0.845 | Ron Zhu |
Related Papers
Detecting Harmful Memes with Decoupled Understanding and Guided CoT Reasoning2025-06-10LLM-based Semantic Augmentation for Harmful Content Detection2025-04-22Robust Adaptation of Large Multimodal Models for Retrieval Augmented Hateful Meme Detection2025-02-18Demystifying Hateful Content: Leveraging Large Multimodal Models for Hateful Meme Detection with Explainable Decisions2025-02-16Figurative-cum-Commonsense Knowledge Infusion for Multimodal Mental Health Meme Classification2025-01-25Prompt-enhanced Network for Hateful Meme Classification2024-11-12MemeCLIP: Leveraging CLIP Representations for Multimodal Meme Classification2024-09-23What Makes a Meme a Meme? Identifying Memes for Memetics-Aware Dataset Creation2024-07-16