TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Papers/Video Instance Segmentation

Video Instance Segmentation

Linjie Yang, Yuchen Fan, Ning Xu

2019-05-12ICCV 2019 10SegmentationSemantic SegmentationInstance SegmentationVideo UnderstandingVideo Instance Segmentation
PaperPDFCodeCodeCodeCode(official)CodeCode

Abstract

In this paper we present a new computer vision task, named video instance segmentation. The goal of this new task is simultaneous detection, segmentation and tracking of instances in videos. In words, it is the first time that the image instance segmentation problem is extended to the video domain. To facilitate research on this new task, we propose a large-scale benchmark called YouTube-VIS, which consists of 2883 high-resolution YouTube videos, a 40-category label set and 131k high-quality instance masks. In addition, we propose a novel algorithm called MaskTrack R-CNN for this task. Our new method introduces a new tracking branch to Mask R-CNN to jointly perform the detection, segmentation and tracking tasks simultaneously. Finally, we evaluate the proposed method and several strong baselines on our new dataset. Experimental results clearly demonstrate the advantages of the proposed algorithm and reveal insight for future improvement. We believe the video instance segmentation task will motivate the community along the line of research for video understanding.

Results

TaskDatasetMetricValueModel
Video Instance SegmentationYouTube-VIS validationAP5051.1MaskTrack R-CNN (ResNet-50, single-scale training and test)
Video Instance SegmentationYouTube-VIS validationAP7532.6MaskTrack R-CNN (ResNet-50, single-scale training and test)
Video Instance SegmentationYouTube-VIS validationAR131MaskTrack R-CNN (ResNet-50, single-scale training and test)
Video Instance SegmentationYouTube-VIS validationAR1035.5MaskTrack R-CNN (ResNet-50, single-scale training and test)
Video Instance SegmentationYouTube-VIS validationmask AP30.3MaskTrack R-CNN (ResNet-50, single-scale training and test)

Related Papers

SeC: Advancing Complex Video Object Segmentation via Progressive Concept Construction2025-07-21Deep Learning-Based Fetal Lung Segmentation from Diffusion-weighted MRI Images and Lung Maturity Evaluation for Fetal Growth Restriction2025-07-17DiffOSeg: Omni Medical Image Segmentation via Multi-Expert Collaboration Diffusion Model2025-07-17From Variability To Accuracy: Conditional Bernoulli Diffusion Models with Consensus-Driven Correction for Thin Structure Segmentation2025-07-17Unleashing Vision Foundation Models for Coronary Artery Segmentation: Parallel ViT-CNN Encoding and Variational Fusion2025-07-17SCORE: Scene Context Matters in Open-Vocabulary Remote Sensing Instance Segmentation2025-07-17Unified Medical Image Segmentation with State Space Modeling Snake2025-07-17A Privacy-Preserving Semantic-Segmentation Method Using Domain-Adaptation Technique2025-07-17