Miquel Martí i Rabadán, Alessandro Pieropan, Hossein Azizpour, Atsuto Maki
We propose Dense FixMatch, a simple method for online semi-supervised learning of dense and structured prediction tasks combining pseudo-labeling and consistency regularization via strong data augmentation. We enable the application of FixMatch in semi-supervised learning problems beyond image classification by adding a matching operation on the pseudo-labels. This allows us to still use the full strength of data augmentation pipelines, including geometric transformations. We evaluate it on semi-supervised semantic segmentation on Cityscapes and Pascal VOC with different percentages of labeled data and ablate design choices and hyper-parameters. Dense FixMatch significantly improves results compared to supervised learning using only labeled data, approaching its performance with 1/4 of the labeled samples.
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Semantic Segmentation | Pascal VOC 2012 6.25% labeled | Validation mIoU | 54.85 | Dense FixMatch (DeepLabv3+ ResNet-101, over-sampling, single pass eval) |
| Semantic Segmentation | Pascal VOC 2012 6.25% labeled | Validation mIoU | 52.15 | Dense FixMatch (DeepLabv3+ ResNet-50, over-sampling, single pass eval) |
| Semantic Segmentation | PASCAL VOC 2012 25% labeled | Validation mIoU | 72.04 | Dense FixMatch (DeepLabv3+ ResNet-101, over-sampling, single pass eval) |
| Semantic Segmentation | PASCAL VOC 2012 25% labeled | Validation mIoU | 69.02 | Dense FixMatch (DeepLabv3+ ResNet-50, over-sampling, single pass eval) |
| Semantic Segmentation | Cityscapes 93 labeled | Validation mIoU | 66.97 | Dense FixMatch (DeepLabv3+ ResNet-101, over-sampling, single pass eval) |
| Semantic Segmentation | Cityscapes 93 labeled | Validation mIoU | 65.81 | Dense FixMatch (DeepLabv3+ ResNet-50, over-sampling, single-pass eval) |
| Semantic Segmentation | Cityscapes with extra (no coarse labels) | Validation mIoU | 80.82 | Dense FixMatch (DeepLabv3+ ResNet-101, over-sampling, single pass eval) |
| Semantic Segmentation | Cityscapes with extra (no coarse labels) | Validation mIoU | 79.98 | Dense FixMatch (DeepLabv3+ ResNet-50, over-sampling, single pass eval) |
| 10-shot image generation | Pascal VOC 2012 6.25% labeled | Validation mIoU | 54.85 | Dense FixMatch (DeepLabv3+ ResNet-101, over-sampling, single pass eval) |
| 10-shot image generation | Pascal VOC 2012 6.25% labeled | Validation mIoU | 52.15 | Dense FixMatch (DeepLabv3+ ResNet-50, over-sampling, single pass eval) |
| 10-shot image generation | PASCAL VOC 2012 25% labeled | Validation mIoU | 72.04 | Dense FixMatch (DeepLabv3+ ResNet-101, over-sampling, single pass eval) |
| 10-shot image generation | PASCAL VOC 2012 25% labeled | Validation mIoU | 69.02 | Dense FixMatch (DeepLabv3+ ResNet-50, over-sampling, single pass eval) |
| 10-shot image generation | Cityscapes 93 labeled | Validation mIoU | 66.97 | Dense FixMatch (DeepLabv3+ ResNet-101, over-sampling, single pass eval) |
| 10-shot image generation | Cityscapes 93 labeled | Validation mIoU | 65.81 | Dense FixMatch (DeepLabv3+ ResNet-50, over-sampling, single-pass eval) |
| 10-shot image generation | Cityscapes with extra (no coarse labels) | Validation mIoU | 80.82 | Dense FixMatch (DeepLabv3+ ResNet-101, over-sampling, single pass eval) |
| 10-shot image generation | Cityscapes with extra (no coarse labels) | Validation mIoU | 79.98 | Dense FixMatch (DeepLabv3+ ResNet-50, over-sampling, single pass eval) |