Bharath Hariharan, Ross Girshick
Low-shot visual learning---the ability to recognize novel object categories from very few examples---is a hallmark of human visual intelligence. Existing machine learning approaches fail to generalize in the same way. To make progress on this foundational problem, we present a low-shot learning benchmark on complex images that mimics challenges faced by recognition systems in the wild. We then propose a) representation regularization techniques, and b) techniques to hallucinate additional training examples for data-starved classes. Together, our methods improve the effectiveness of convolutional networks in low-shot learning, improving the one-shot accuracy on novel classes by 2.3x on the challenging ImageNet dataset.
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Image Classification | ImageNet-FS (5-shot, all) | Top-5 Accuracy (%) | 77.4 | SGM (ResNet-50) |
| Image Classification | ImageNet-FS (5-shot, all) | Top-5 Accuracy (%) | 77.3 | SGM w/ G (ResNet-50) |
| Image Classification | ImageNet-FS (1-shot, novel) | Top-5 Accuracy (%) | 52.9 | SGM (ResNet-50) |
| Image Classification | ImageNet-FS (2-shot, novel) | Top-5 Accuracy (%) | 67 | SGM (ResNet-50) |
| Image Classification | ImageNet-FS (2-shot, novel) | Top-5 Accuracy (%) | 64.9 | SGM w/G (ResNet-50) |
| Few-Shot Image Classification | ImageNet-FS (5-shot, all) | Top-5 Accuracy (%) | 77.4 | SGM (ResNet-50) |
| Few-Shot Image Classification | ImageNet-FS (5-shot, all) | Top-5 Accuracy (%) | 77.3 | SGM w/ G (ResNet-50) |
| Few-Shot Image Classification | ImageNet-FS (1-shot, novel) | Top-5 Accuracy (%) | 52.9 | SGM (ResNet-50) |
| Few-Shot Image Classification | ImageNet-FS (2-shot, novel) | Top-5 Accuracy (%) | 67 | SGM (ResNet-50) |
| Few-Shot Image Classification | ImageNet-FS (2-shot, novel) | Top-5 Accuracy (%) | 64.9 | SGM w/G (ResNet-50) |