TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Papers/STC: Spatio-Temporal Contrastive Learning for Video Instan...

STC: Spatio-Temporal Contrastive Learning for Video Instance Segmentation

Zhengkai Jiang, Zhangxuan Gu, Jinlong Peng, Hang Zhou, Liang Liu, Yabiao Wang, Ying Tai, Chengjie Wang, Liqing Zhang

2022-02-08SegmentationSemantic SegmentationContrastive LearningInstance SegmentationVideo Instance Segmentation
PaperPDF

Abstract

Video Instance Segmentation (VIS) is a task that simultaneously requires classification, segmentation, and instance association in a video. Recent VIS approaches rely on sophisticated pipelines to achieve this goal, including RoI-related operations or 3D convolutions. In contrast, we present a simple and efficient single-stage VIS framework based on the instance segmentation method CondInst by adding an extra tracking head. To improve instance association accuracy, a novel bi-directional spatio-temporal contrastive learning strategy for tracking embedding across frames is proposed. Moreover, an instance-wise temporal consistency scheme is utilized to produce temporally coherent results. Experiments conducted on the YouTube-VIS-2019, YouTube-VIS-2021, and OVIS-2021 datasets validate the effectiveness and efficiency of the proposed method. We hope the proposed framework can serve as a simple and strong alternative for many other instance-level video association tasks.

Results

TaskDatasetMetricValueModel
Video Instance SegmentationYouTube-VIS validationAP5057.2STC (ResNet-50)
Video Instance SegmentationYouTube-VIS validationAP7538.6STC (ResNet-50)
Video Instance SegmentationYouTube-VIS validationAR136.9STC (ResNet-50)
Video Instance SegmentationYouTube-VIS validationAR1044.5STC (ResNet-50)
Video Instance SegmentationYouTube-VIS validationmask AP36.7STC (ResNet-50)
Video Instance SegmentationOVIS validationAP5033.5STC (ResNet-50)
Video Instance SegmentationOVIS validationAP7513.4STC (ResNet-50)
Video Instance SegmentationOVIS validationmask AP15.5STC (ResNet-50)

Related Papers

SeC: Advancing Complex Video Object Segmentation via Progressive Concept Construction2025-07-21Deep Learning-Based Fetal Lung Segmentation from Diffusion-weighted MRI Images and Lung Maturity Evaluation for Fetal Growth Restriction2025-07-17DiffOSeg: Omni Medical Image Segmentation via Multi-Expert Collaboration Diffusion Model2025-07-17From Variability To Accuracy: Conditional Bernoulli Diffusion Models with Consensus-Driven Correction for Thin Structure Segmentation2025-07-17Unleashing Vision Foundation Models for Coronary Artery Segmentation: Parallel ViT-CNN Encoding and Variational Fusion2025-07-17SCORE: Scene Context Matters in Open-Vocabulary Remote Sensing Instance Segmentation2025-07-17Unified Medical Image Segmentation with State Space Modeling Snake2025-07-17A Privacy-Preserving Semantic-Segmentation Method Using Domain-Adaptation Technique2025-07-17