TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Papers/Occluded Video Instance Segmentation: A Benchmark

Occluded Video Instance Segmentation: A Benchmark

Jiyang Qi, Yan Gao, Yao Hu, Xinggang Wang, Xiaoyu Liu, Xiang Bai, Serge Belongie, Alan Yuille, Philip H. S. Torr, Song Bai

2021-02-02SegmentationSemantic SegmentationInstance SegmentationVideo UnderstandingVideo Instance Segmentation
PaperPDFCodeCode

Abstract

Can our video understanding systems perceive objects when a heavy occlusion exists in a scene? To answer this question, we collect a large-scale dataset called OVIS for occluded video instance segmentation, that is, to simultaneously detect, segment, and track instances in occluded scenes. OVIS consists of 296k high-quality instance masks from 25 semantic categories, where object occlusions usually occur. While our human vision systems can understand those occluded instances by contextual reasoning and association, our experiments suggest that current video understanding systems cannot. On the OVIS dataset, the highest AP achieved by state-of-the-art algorithms is only 16.3, which reveals that we are still at a nascent stage for understanding objects, instances, and videos in a real-world scenario. We also present a simple plug-and-play module that performs temporal feature calibration to complement missing object cues caused by occlusion. Built upon MaskTrack R-CNN and SipMask, we obtain a remarkable AP improvement on the OVIS dataset. The OVIS dataset and project code are available at http://songbai.site/ovis .

Results

TaskDatasetMetricValueModel
Video Instance SegmentationYouTube-VIS validationAP5055.6CSipMask
Video Instance SegmentationYouTube-VIS validationAP7538.1CSipMask
Video Instance SegmentationYouTube-VIS validationmask AP35.1CSipMask
Video Instance SegmentationYouTube-VIS validationAP5052.8CMaskTrack R-CNN
Video Instance SegmentationYouTube-VIS validationAP7534.9CMaskTrack R-CNN
Video Instance SegmentationYouTube-VIS validationmask AP32.1CMaskTrack R-CNN
Video Instance SegmentationOVIS validationAP5033.9CMaskTrack R-CNN (ResNet-50)
Video Instance SegmentationOVIS validationAP7513.1CMaskTrack R-CNN (ResNet-50)
Video Instance SegmentationOVIS validationAPho4.1CMaskTrack R-CNN (ResNet-50)
Video Instance SegmentationOVIS validationAPmo18.7CMaskTrack R-CNN (ResNet-50)
Video Instance SegmentationOVIS validationAPso28.6CMaskTrack R-CNN (ResNet-50)
Video Instance SegmentationOVIS validationmask AP15.4CMaskTrack R-CNN (ResNet-50)
Video Instance SegmentationOVIS validationAP5029.9CSipMask (ResNet-50)
Video Instance SegmentationOVIS validationAP7512.5CSipMask (ResNet-50)
Video Instance SegmentationOVIS validationAPho2.7CSipMask (ResNet-50)
Video Instance SegmentationOVIS validationAPmo12.8CSipMask (ResNet-50)
Video Instance SegmentationOVIS validationAPso23CSipMask (ResNet-50)
Video Instance SegmentationOVIS validationmask AP14.3CSipMask (ResNet-50)

Related Papers

SeC: Advancing Complex Video Object Segmentation via Progressive Concept Construction2025-07-21Deep Learning-Based Fetal Lung Segmentation from Diffusion-weighted MRI Images and Lung Maturity Evaluation for Fetal Growth Restriction2025-07-17DiffOSeg: Omni Medical Image Segmentation via Multi-Expert Collaboration Diffusion Model2025-07-17From Variability To Accuracy: Conditional Bernoulli Diffusion Models with Consensus-Driven Correction for Thin Structure Segmentation2025-07-17Unleashing Vision Foundation Models for Coronary Artery Segmentation: Parallel ViT-CNN Encoding and Variational Fusion2025-07-17SCORE: Scene Context Matters in Open-Vocabulary Remote Sensing Instance Segmentation2025-07-17Unified Medical Image Segmentation with State Space Modeling Snake2025-07-17A Privacy-Preserving Semantic-Segmentation Method Using Domain-Adaptation Technique2025-07-17