Yi Li, Zhanghui Kuang, Liyang Liu, Yimin Chen, Wayne Zhang
Most weakly supervised semantic segmentation (WSSS) methods follow the pipeline that generates pseudo-masks initially and trains the segmentation model with the pseudo-masks in fully supervised manner after. However, we find some matters related to the pseudo-masks, including high quality pseudo-masks generation from class activation maps (CAMs), and training with noisy pseudo-mask supervision. For these matters, we propose the following designs to push the performance to new state-of-art: (i) Coefficient of Variation Smoothing to smooth the CAMs adaptively; (ii) Proportional Pseudo-mask Generation to project the expanded CAMs to pseudo-mask based on a new metric indicating the importance of each class on each location, instead of the scores trained from binary classifiers. (iii) Pretended Under-Fitting strategy to suppress the influence of noise in pseudo-mask; (iv) Cyclic Pseudo-mask to boost the pseudo-masks during training of fully supervised semantic segmentation (FSSS). Experiments based on our methods achieve new state-of-art results on two changeling weakly supervised semantic segmentation datasets, pushing the mIoU to 70.0% and 40.2% on PAS-CAL VOC 2012 and MS COCO 2014 respectively. Codes including segmentation framework are released at https://github.com/Eli-YiLi/PMM
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Semantic Segmentation | COCO 2014 val | mIoU | 40.2 | PMM(ScaleNet101, no saliency, no RW) |
| Semantic Segmentation | COCO 2014 val | mIoU | 36.7 | PMM(ResNet38, no saliency, no RW) |
| Semantic Segmentation | PASCAL VOC 2012 val | Mean IoU | 70 | PMM(Res2Net101, no saliency, no RW) |
| Semantic Segmentation | PASCAL VOC 2012 val | Mean IoU | 68.5 | PMM(ResNet38) |
| Semantic Segmentation | PASCAL VOC 2012 val | Mean IoU | 68.5 | PMM(ResNet38, no saliency, no RW) |
| Semantic Segmentation | PASCAL VOC 2012 test | Mean IoU | 70.5 | PMM(Res2Net101, no saliency, no RW) |
| Semantic Segmentation | PASCAL VOC 2012 test | Mean IoU | 69 | PMM(ResNet38, no saliency, no RW) |
| 10-shot image generation | COCO 2014 val | mIoU | 40.2 | PMM(ScaleNet101, no saliency, no RW) |
| 10-shot image generation | COCO 2014 val | mIoU | 36.7 | PMM(ResNet38, no saliency, no RW) |
| 10-shot image generation | PASCAL VOC 2012 val | Mean IoU | 70 | PMM(Res2Net101, no saliency, no RW) |
| 10-shot image generation | PASCAL VOC 2012 val | Mean IoU | 68.5 | PMM(ResNet38) |
| 10-shot image generation | PASCAL VOC 2012 val | Mean IoU | 68.5 | PMM(ResNet38, no saliency, no RW) |
| 10-shot image generation | PASCAL VOC 2012 test | Mean IoU | 70.5 | PMM(Res2Net101, no saliency, no RW) |
| 10-shot image generation | PASCAL VOC 2012 test | Mean IoU | 69 | PMM(ResNet38, no saliency, no RW) |