Yifan Zhang, Bryan Hooi, Lanqing Hong, Jiashi Feng
Existing long-tailed recognition methods, aiming to train class-balanced models from long-tailed data, generally assume the models would be evaluated on the uniform test class distribution. However, practical test class distributions often violate this assumption (e.g., being either long-tailed or even inversely long-tailed), which may lead existing methods to fail in real applications. In this paper, we study a more practical yet challenging task, called test-agnostic long-tailed recognition, where the training class distribution is long-tailed while the test class distribution is agnostic and not necessarily uniform. In addition to the issue of class imbalance, this task poses another challenge: the class distribution shift between the training and test data is unknown. To tackle this task, we propose a novel approach, called Self-supervised Aggregation of Diverse Experts, which consists of two strategies: (i) a new skill-diverse expert learning strategy that trains multiple experts from a single and stationary long-tailed dataset to separately handle different class distributions; (ii) a novel test-time expert aggregation strategy that leverages self-supervision to aggregate the learned multiple experts for handling unknown test class distributions. We theoretically show that our self-supervised strategy has a provable ability to simulate test-agnostic class distributions. Promising empirical results demonstrate the effectiveness of our method on both vanilla and test-agnostic long-tailed recognition. Code is available at \url{https://github.com/Vanint/SADE-AgnosticLT}.
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Image Classification | Places-LT | Top 1 Accuracy | 40.9 | TADE |
| Image Classification | Places-LT | Top-1 Accuracy | 41.3 | TADE |
| Image Classification | CIFAR-10-LT (ρ=10) | Error Rate | 9.2 | TADE |
| Image Classification | CIFAR-10-LT (ρ=10) | Error Rate | 10.3 | RIDE |
| Image Classification | CIFAR-100-LT (ρ=50) | Error Rate | 46.1 | TADE |
| Image Classification | CIFAR-100-LT (ρ=10) | Error Rate | 36.4 | TADE |
| Image Classification | ImageNet-LT | Top-1 Accuracy | 61.4 | TADE(ResNeXt101-32x4d) |
| Image Classification | ImageNet-LT | Top-1 Accuracy | 58.8 | TADE(ResNeXt-50) |
| Image Classification | CIFAR-100-LT (ρ=100) | Error Rate | 50.2 | TADE |
| Image Classification | CIFAR-10-LT (ρ=100) | Error Rate | 16.2 | TADE |
| Few-Shot Image Classification | Places-LT | Top 1 Accuracy | 40.9 | TADE |
| Few-Shot Image Classification | Places-LT | Top-1 Accuracy | 41.3 | TADE |
| Few-Shot Image Classification | CIFAR-10-LT (ρ=10) | Error Rate | 9.2 | TADE |
| Few-Shot Image Classification | CIFAR-10-LT (ρ=10) | Error Rate | 10.3 | RIDE |
| Few-Shot Image Classification | CIFAR-100-LT (ρ=50) | Error Rate | 46.1 | TADE |
| Few-Shot Image Classification | CIFAR-100-LT (ρ=10) | Error Rate | 36.4 | TADE |
| Few-Shot Image Classification | ImageNet-LT | Top-1 Accuracy | 61.4 | TADE(ResNeXt101-32x4d) |
| Few-Shot Image Classification | ImageNet-LT | Top-1 Accuracy | 58.8 | TADE(ResNeXt-50) |
| Few-Shot Image Classification | CIFAR-100-LT (ρ=100) | Error Rate | 50.2 | TADE |
| Few-Shot Image Classification | CIFAR-10-LT (ρ=100) | Error Rate | 16.2 | TADE |
| Generalized Few-Shot Classification | Places-LT | Top 1 Accuracy | 40.9 | TADE |
| Generalized Few-Shot Classification | Places-LT | Top-1 Accuracy | 41.3 | TADE |
| Generalized Few-Shot Classification | CIFAR-10-LT (ρ=10) | Error Rate | 9.2 | TADE |
| Generalized Few-Shot Classification | CIFAR-10-LT (ρ=10) | Error Rate | 10.3 | RIDE |
| Generalized Few-Shot Classification | CIFAR-100-LT (ρ=50) | Error Rate | 46.1 | TADE |
| Generalized Few-Shot Classification | CIFAR-100-LT (ρ=10) | Error Rate | 36.4 | TADE |
| Generalized Few-Shot Classification | ImageNet-LT | Top-1 Accuracy | 61.4 | TADE(ResNeXt101-32x4d) |
| Generalized Few-Shot Classification | ImageNet-LT | Top-1 Accuracy | 58.8 | TADE(ResNeXt-50) |
| Generalized Few-Shot Classification | CIFAR-100-LT (ρ=100) | Error Rate | 50.2 | TADE |
| Generalized Few-Shot Classification | CIFAR-10-LT (ρ=100) | Error Rate | 16.2 | TADE |
| Long-tail Learning | Places-LT | Top 1 Accuracy | 40.9 | TADE |
| Long-tail Learning | Places-LT | Top-1 Accuracy | 41.3 | TADE |
| Long-tail Learning | CIFAR-10-LT (ρ=10) | Error Rate | 9.2 | TADE |
| Long-tail Learning | CIFAR-10-LT (ρ=10) | Error Rate | 10.3 | RIDE |
| Long-tail Learning | CIFAR-100-LT (ρ=50) | Error Rate | 46.1 | TADE |
| Long-tail Learning | CIFAR-100-LT (ρ=10) | Error Rate | 36.4 | TADE |
| Long-tail Learning | ImageNet-LT | Top-1 Accuracy | 61.4 | TADE(ResNeXt101-32x4d) |
| Long-tail Learning | ImageNet-LT | Top-1 Accuracy | 58.8 | TADE(ResNeXt-50) |
| Long-tail Learning | CIFAR-100-LT (ρ=100) | Error Rate | 50.2 | TADE |
| Long-tail Learning | CIFAR-10-LT (ρ=100) | Error Rate | 16.2 | TADE |
| Generalized Few-Shot Learning | Places-LT | Top 1 Accuracy | 40.9 | TADE |
| Generalized Few-Shot Learning | Places-LT | Top-1 Accuracy | 41.3 | TADE |
| Generalized Few-Shot Learning | CIFAR-10-LT (ρ=10) | Error Rate | 9.2 | TADE |
| Generalized Few-Shot Learning | CIFAR-10-LT (ρ=10) | Error Rate | 10.3 | RIDE |
| Generalized Few-Shot Learning | CIFAR-100-LT (ρ=50) | Error Rate | 46.1 | TADE |
| Generalized Few-Shot Learning | CIFAR-100-LT (ρ=10) | Error Rate | 36.4 | TADE |
| Generalized Few-Shot Learning | ImageNet-LT | Top-1 Accuracy | 61.4 | TADE(ResNeXt101-32x4d) |
| Generalized Few-Shot Learning | ImageNet-LT | Top-1 Accuracy | 58.8 | TADE(ResNeXt-50) |
| Generalized Few-Shot Learning | CIFAR-100-LT (ρ=100) | Error Rate | 50.2 | TADE |
| Generalized Few-Shot Learning | CIFAR-10-LT (ρ=100) | Error Rate | 16.2 | TADE |