TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Papers/Action Segmentation with Mixed Temporal Domain Adaptation

Action Segmentation with Mixed Temporal Domain Adaptation

Min-Hung Chen, Baopu Li, Yingze Bao, Ghassan AlRegib

2021-04-15Action SegmentationDomain Adaptation
PaperPDF

Abstract

The main progress for action segmentation comes from densely-annotated data for fully-supervised learning. Since manual annotation for frame-level actions is time-consuming and challenging, we propose to exploit auxiliary unlabeled videos, which are much easier to obtain, by shaping this problem as a domain adaptation (DA) problem. Although various DA techniques have been proposed in recent years, most of them have been developed only for the spatial direction. Therefore, we propose Mixed Temporal Domain Adaptation (MTDA) to jointly align frame- and video-level embedded feature spaces across domains, and further integrate with the domain attention mechanism to focus on aligning the frame-level features with higher domain discrepancy, leading to more effective domain adaptation. Finally, we evaluate our proposed methods on three challenging datasets (GTEA, 50Salads, and Breakfast), and validate that MTDA outperforms the current state-of-the-art methods on all three datasets by large margins (e.g. 6.4% gain on F1@50 and 6.8% gain on the edit score for GTEA).

Results

TaskDatasetMetricValueModel
Action Localization50 SaladsAcc83.2DA
Action Localization50 SaladsEdit75.2DA
Action Localization50 SaladsF1@10%82DA
Action Localization50 SaladsF1@25%80.1DA
Action Localization50 SaladsF1@50%72.5DA
Action LocalizationGTEAAcc80DA
Action LocalizationGTEAEdit85.8DA
Action LocalizationGTEAF1@10%90.5DA
Action LocalizationGTEAF1@25%88.4DA
Action LocalizationGTEAF1@50%76.2DA
Action LocalizationBreakfastAcc71DA
Action LocalizationBreakfastAverage F166.4DA
Action LocalizationBreakfastEdit73.6DA
Action LocalizationBreakfastF1@10%74.2DA
Action LocalizationBreakfastF1@25%68.6DA
Action LocalizationBreakfastF1@50%56.5DA
Action Segmentation50 SaladsAcc83.2DA
Action Segmentation50 SaladsEdit75.2DA
Action Segmentation50 SaladsF1@10%82DA
Action Segmentation50 SaladsF1@25%80.1DA
Action Segmentation50 SaladsF1@50%72.5DA
Action SegmentationGTEAAcc80DA
Action SegmentationGTEAEdit85.8DA
Action SegmentationGTEAF1@10%90.5DA
Action SegmentationGTEAF1@25%88.4DA
Action SegmentationGTEAF1@50%76.2DA
Action SegmentationBreakfastAcc71DA
Action SegmentationBreakfastAverage F166.4DA
Action SegmentationBreakfastEdit73.6DA
Action SegmentationBreakfastF1@10%74.2DA
Action SegmentationBreakfastF1@25%68.6DA
Action SegmentationBreakfastF1@50%56.5DA

Related Papers

A Privacy-Preserving Semantic-Segmentation Method Using Domain-Adaptation Technique2025-07-17Domain Borders Are There to Be Crossed With Federated Few-Shot Adaptation2025-07-14Self-supervised pretraining of vision transformers for animal behavioral analysis and neural encoding2025-07-13An Offline Mobile Conversational Agent for Mental Health Support: Learning from Emotional Dialogues and Psychological Texts with Student-Centered Evaluation2025-07-11The Bayesian Approach to Continual Learning: An Overview2025-07-11Doodle Your Keypoints: Sketch-Based Few-Shot Keypoint Detection2025-07-10YOLO-APD: Enhancing YOLOv8 for Robust Pedestrian Detection on Complex Road Geometries2025-07-07CORE-ReID V2: Advancing the Domain Adaptation for Object Re-Identification with Optimized Training and Ensemble Fusion2025-07-04