Jie Liu, Yanqi Bao, Wenzhe Yin, Haochen Wang, Yang Gao, Jan-Jakob Sonke, Efstratios Gavves
Few-shot semantic segmentation (FSS) aims to achieve novel objects segmentation with only a few annotated samples and has made great progress recently. Most of the existing FSS models focus on the feature matching between support and query to tackle FSS. However, the appearance variations between objects from the same category could be extremely large, leading to unreliable feature matching and query mask prediction. To this end, we propose a Support-induced Graph Convolutional Network (SiGCN) to explicitly excavate latent context structure in query images. Specifically, we propose a Support-induced Graph Reasoning (SiGR) module to capture salient query object parts at different semantic levels with a Support-induced GCN. Furthermore, an instance association (IA) module is designed to capture high-order instance context from both support and query instances. By integrating the proposed two modules, SiGCN can learn rich query context representation, and thus being more robust to appearance variations. Extensive experiments on PASCAL-5i and COCO-20i demonstrate that our SiGCN achieves state-of-the-art performance.
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Few-Shot Learning | COCO-20i (5-shot) | FB-IoU | 66.2 | SiGCN (ResNet-50) |
| Few-Shot Learning | COCO-20i (5-shot) | Mean IoU | 48 | SiGCN (ResNet-50) |
| Few-Shot Learning | PASCAL-5i (1-Shot) | FB-IoU | 78.3 | SiGCN (ResNet-101) |
| Few-Shot Learning | PASCAL-5i (1-Shot) | Mean IoU | 65.7 | SiGCN (ResNet-101) |
| Few-Shot Learning | PASCAL-5i (1-Shot) | FB-IoU | 77.5 | SiGCN (ResNet-50) |
| Few-Shot Learning | PASCAL-5i (1-Shot) | Mean IoU | 65.3 | SiGCN (ResNet-50) |
| Few-Shot Learning | PASCAL-5i (1-Shot) | FB-IoU | 73.5 | SiGCN (VGG-16) |
| Few-Shot Learning | PASCAL-5i (1-Shot) | Mean IoU | 60.8 | SiGCN (VGG-16) |
| Few-Shot Learning | COCO-20i (1-shot) | FB-IoU | 62.7 | SiGCN (ResNet-50) |
| Few-Shot Learning | COCO-20i (1-shot) | Mean IoU | 41.4 | SiGCN (ResNet-50) |
| Few-Shot Learning | PASCAL-5i (5-Shot) | FB-IoU | 78.3 | SiGCN (ResNet-50) |
| Few-Shot Learning | PASCAL-5i (5-Shot) | Mean IoU | 68.5 | SiGCN (ResNet-50) |
| Few-Shot Semantic Segmentation | COCO-20i (5-shot) | FB-IoU | 66.2 | SiGCN (ResNet-50) |
| Few-Shot Semantic Segmentation | COCO-20i (5-shot) | Mean IoU | 48 | SiGCN (ResNet-50) |
| Few-Shot Semantic Segmentation | PASCAL-5i (1-Shot) | FB-IoU | 78.3 | SiGCN (ResNet-101) |
| Few-Shot Semantic Segmentation | PASCAL-5i (1-Shot) | Mean IoU | 65.7 | SiGCN (ResNet-101) |
| Few-Shot Semantic Segmentation | PASCAL-5i (1-Shot) | FB-IoU | 77.5 | SiGCN (ResNet-50) |
| Few-Shot Semantic Segmentation | PASCAL-5i (1-Shot) | Mean IoU | 65.3 | SiGCN (ResNet-50) |
| Few-Shot Semantic Segmentation | PASCAL-5i (1-Shot) | FB-IoU | 73.5 | SiGCN (VGG-16) |
| Few-Shot Semantic Segmentation | PASCAL-5i (1-Shot) | Mean IoU | 60.8 | SiGCN (VGG-16) |
| Few-Shot Semantic Segmentation | COCO-20i (1-shot) | FB-IoU | 62.7 | SiGCN (ResNet-50) |
| Few-Shot Semantic Segmentation | COCO-20i (1-shot) | Mean IoU | 41.4 | SiGCN (ResNet-50) |
| Few-Shot Semantic Segmentation | PASCAL-5i (5-Shot) | FB-IoU | 78.3 | SiGCN (ResNet-50) |
| Few-Shot Semantic Segmentation | PASCAL-5i (5-Shot) | Mean IoU | 68.5 | SiGCN (ResNet-50) |
| Meta-Learning | COCO-20i (5-shot) | FB-IoU | 66.2 | SiGCN (ResNet-50) |
| Meta-Learning | COCO-20i (5-shot) | Mean IoU | 48 | SiGCN (ResNet-50) |
| Meta-Learning | PASCAL-5i (1-Shot) | FB-IoU | 78.3 | SiGCN (ResNet-101) |
| Meta-Learning | PASCAL-5i (1-Shot) | Mean IoU | 65.7 | SiGCN (ResNet-101) |
| Meta-Learning | PASCAL-5i (1-Shot) | FB-IoU | 77.5 | SiGCN (ResNet-50) |
| Meta-Learning | PASCAL-5i (1-Shot) | Mean IoU | 65.3 | SiGCN (ResNet-50) |
| Meta-Learning | PASCAL-5i (1-Shot) | FB-IoU | 73.5 | SiGCN (VGG-16) |
| Meta-Learning | PASCAL-5i (1-Shot) | Mean IoU | 60.8 | SiGCN (VGG-16) |
| Meta-Learning | COCO-20i (1-shot) | FB-IoU | 62.7 | SiGCN (ResNet-50) |
| Meta-Learning | COCO-20i (1-shot) | Mean IoU | 41.4 | SiGCN (ResNet-50) |
| Meta-Learning | PASCAL-5i (5-Shot) | FB-IoU | 78.3 | SiGCN (ResNet-50) |
| Meta-Learning | PASCAL-5i (5-Shot) | Mean IoU | 68.5 | SiGCN (ResNet-50) |