Seulki Park, Youngkyu Hong, Byeongho Heo, Sangdoo Yun, Jin Young Choi
The problem of class imbalanced data is that the generalization performance of the classifier deteriorates due to the lack of data from minority classes. In this paper, we propose a novel minority over-sampling method to augment diversified minority samples by leveraging the rich context of the majority classes as background images. To diversify the minority samples, our key idea is to paste an image from a minority class onto rich-context images from a majority class, using them as background images. Our method is simple and can be easily combined with the existing long-tailed recognition methods. We empirically prove the effectiveness of the proposed oversampling method through extensive experiments and ablation studies. Without any architectural changes or complex algorithms, our method achieves state-of-the-art performance on various long-tailed classification benchmarks. Our code is made available at https://github.com/naver-ai/cmo.
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Image Classification | ImageNet-LT | Top-1 Accuracy | 58 | BS-CMO (ResNet-50) |
| Image Classification | CIFAR-100-LT (ρ=100) | Error Rate | 50 | RIDE 3 experts + CMO |
| Image Classification | CIFAR-100-LT (ρ=100) | Error Rate | 52.8 | LDAM-DRW + CMO |
| Image Classification | CIFAR-100-LT (ρ=100) | Error Rate | 53.4 | Balanced Softmax + CMO |
| Image Classification | CIFAR-100-LT (ρ=100) | Error Rate | 58.9 | CE-DRW |
| Few-Shot Image Classification | ImageNet-LT | Top-1 Accuracy | 58 | BS-CMO (ResNet-50) |
| Few-Shot Image Classification | CIFAR-100-LT (ρ=100) | Error Rate | 50 | RIDE 3 experts + CMO |
| Few-Shot Image Classification | CIFAR-100-LT (ρ=100) | Error Rate | 52.8 | LDAM-DRW + CMO |
| Few-Shot Image Classification | CIFAR-100-LT (ρ=100) | Error Rate | 53.4 | Balanced Softmax + CMO |
| Few-Shot Image Classification | CIFAR-100-LT (ρ=100) | Error Rate | 58.9 | CE-DRW |
| Generalized Few-Shot Classification | ImageNet-LT | Top-1 Accuracy | 58 | BS-CMO (ResNet-50) |
| Generalized Few-Shot Classification | CIFAR-100-LT (ρ=100) | Error Rate | 50 | RIDE 3 experts + CMO |
| Generalized Few-Shot Classification | CIFAR-100-LT (ρ=100) | Error Rate | 52.8 | LDAM-DRW + CMO |
| Generalized Few-Shot Classification | CIFAR-100-LT (ρ=100) | Error Rate | 53.4 | Balanced Softmax + CMO |
| Generalized Few-Shot Classification | CIFAR-100-LT (ρ=100) | Error Rate | 58.9 | CE-DRW |
| Long-tail Learning | ImageNet-LT | Top-1 Accuracy | 58 | BS-CMO (ResNet-50) |
| Long-tail Learning | CIFAR-100-LT (ρ=100) | Error Rate | 50 | RIDE 3 experts + CMO |
| Long-tail Learning | CIFAR-100-LT (ρ=100) | Error Rate | 52.8 | LDAM-DRW + CMO |
| Long-tail Learning | CIFAR-100-LT (ρ=100) | Error Rate | 53.4 | Balanced Softmax + CMO |
| Long-tail Learning | CIFAR-100-LT (ρ=100) | Error Rate | 58.9 | CE-DRW |
| Generalized Few-Shot Learning | ImageNet-LT | Top-1 Accuracy | 58 | BS-CMO (ResNet-50) |
| Generalized Few-Shot Learning | CIFAR-100-LT (ρ=100) | Error Rate | 50 | RIDE 3 experts + CMO |
| Generalized Few-Shot Learning | CIFAR-100-LT (ρ=100) | Error Rate | 52.8 | LDAM-DRW + CMO |
| Generalized Few-Shot Learning | CIFAR-100-LT (ρ=100) | Error Rate | 53.4 | Balanced Softmax + CMO |
| Generalized Few-Shot Learning | CIFAR-100-LT (ρ=100) | Error Rate | 58.9 | CE-DRW |