TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Papers/Temporal Action Segmentation from Timestamp Supervision

Temporal Action Segmentation from Timestamp Supervision

Zhe Li, Yazan Abu Farha, Juergen Gall

2021-03-11CVPR 2021 1Action SegmentationWeakly Supervised Action LocalizationTemporal Action SegmentationSegmentation
PaperPDFCode(official)

Abstract

Temporal action segmentation approaches have been very successful recently. However, annotating videos with frame-wise labels to train such models is very expensive and time consuming. While weakly supervised methods trained using only ordered action lists require less annotation effort, the performance is still worse than fully supervised approaches. In this paper, we propose to use timestamp supervision for the temporal action segmentation task. Timestamps require a comparable annotation effort to weakly supervised approaches, and yet provide a more supervisory signal. To demonstrate the effectiveness of timestamp supervision, we propose an approach to train a segmentation model using only timestamps annotations. Our approach uses the model output and the annotated timestamps to generate frame-wise labels by detecting the action changes. We further introduce a confidence loss that forces the predicted probabilities to monotonically decrease as the distance to the timestamps increases. This ensures that all and not only the most distinctive frames of an action are learned during training. The evaluation on four datasets shows that models trained with timestamps annotations achieve comparable performance to the fully supervised approaches.

Results

TaskDatasetMetricValueModel
VideoGTEAmAP@0.1:0.736.4Li et al.
VideoGTEAmAP@0.528.8Li et al.
VideoBEOIDmAP@0.1:0.734.4Li et al.
VideoBEOIDmAP@0.520.3Li et al.
Temporal Action LocalizationGTEAmAP@0.1:0.736.4Li et al.
Temporal Action LocalizationGTEAmAP@0.528.8Li et al.
Temporal Action LocalizationBEOIDmAP@0.1:0.734.4Li et al.
Temporal Action LocalizationBEOIDmAP@0.520.3Li et al.
Zero-Shot LearningGTEAmAP@0.1:0.736.4Li et al.
Zero-Shot LearningGTEAmAP@0.528.8Li et al.
Zero-Shot LearningBEOIDmAP@0.1:0.734.4Li et al.
Zero-Shot LearningBEOIDmAP@0.520.3Li et al.
Action LocalizationGTEAmAP@0.1:0.736.4Li et al.
Action LocalizationGTEAmAP@0.528.8Li et al.
Action LocalizationBEOIDmAP@0.1:0.734.4Li et al.
Action LocalizationBEOIDmAP@0.520.3Li et al.
Weakly Supervised Action LocalizationGTEAmAP@0.1:0.736.4Li et al.
Weakly Supervised Action LocalizationGTEAmAP@0.528.8Li et al.
Weakly Supervised Action LocalizationBEOIDmAP@0.1:0.734.4Li et al.
Weakly Supervised Action LocalizationBEOIDmAP@0.520.3Li et al.

Related Papers

SeC: Advancing Complex Video Object Segmentation via Progressive Concept Construction2025-07-21Deep Learning-Based Fetal Lung Segmentation from Diffusion-weighted MRI Images and Lung Maturity Evaluation for Fetal Growth Restriction2025-07-17DiffOSeg: Omni Medical Image Segmentation via Multi-Expert Collaboration Diffusion Model2025-07-17From Variability To Accuracy: Conditional Bernoulli Diffusion Models with Consensus-Driven Correction for Thin Structure Segmentation2025-07-17Unleashing Vision Foundation Models for Coronary Artery Segmentation: Parallel ViT-CNN Encoding and Variational Fusion2025-07-17SCORE: Scene Context Matters in Open-Vocabulary Remote Sensing Instance Segmentation2025-07-17Unified Medical Image Segmentation with State Space Modeling Snake2025-07-17A Privacy-Preserving Semantic-Segmentation Method Using Domain-Adaptation Technique2025-07-17