TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Papers/Tracking Emerges by Colorizing Videos

Tracking Emerges by Colorizing Videos

Carl Vondrick, Abhinav Shrivastava, Alireza Fathi, Sergio Guadarrama, Kevin Murphy

2018-06-25ECCV 2018 9Visual TrackingOptical Flow EstimationSkeleton Based Action RecognitionColorization
PaperPDFCode

Abstract

We use large amounts of unlabeled video to learn models for visual tracking without manual human supervision. We leverage the natural temporal coherency of color to create a model that learns to colorize gray-scale videos by copying colors from a reference frame. Quantitative and qualitative experiments suggest that this task causes the model to automatically learn to track visual regions. Although the model is trained without any ground-truth labels, our method learns to track well enough to outperform the latest methods based on optical flow. Moreover, our results suggest that failures to track are correlated with failures to colorize, indicating that advancing video colorization may further improve self-supervised visual tracking.

Results

TaskDatasetMetricValueModel
VideoJHMDB Pose TrackingPCK@0.145.2ColorPointer
VideoJHMDB Pose TrackingPCK@0.269.6ColorPointer
VideoJHMDB Pose TrackingPCK@0.380.8ColorPointer
VideoJHMDB Pose TrackingPCK@0.487.5ColorPointer
VideoJHMDB Pose TrackingPCK@0.591.4ColorPointer
Temporal Action LocalizationJHMDB Pose TrackingPCK@0.145.2ColorPointer
Temporal Action LocalizationJHMDB Pose TrackingPCK@0.269.6ColorPointer
Temporal Action LocalizationJHMDB Pose TrackingPCK@0.380.8ColorPointer
Temporal Action LocalizationJHMDB Pose TrackingPCK@0.487.5ColorPointer
Temporal Action LocalizationJHMDB Pose TrackingPCK@0.591.4ColorPointer
Zero-Shot LearningJHMDB Pose TrackingPCK@0.145.2ColorPointer
Zero-Shot LearningJHMDB Pose TrackingPCK@0.269.6ColorPointer
Zero-Shot LearningJHMDB Pose TrackingPCK@0.380.8ColorPointer
Zero-Shot LearningJHMDB Pose TrackingPCK@0.487.5ColorPointer
Zero-Shot LearningJHMDB Pose TrackingPCK@0.591.4ColorPointer
Activity RecognitionJHMDB Pose TrackingPCK@0.145.2ColorPointer
Activity RecognitionJHMDB Pose TrackingPCK@0.269.6ColorPointer
Activity RecognitionJHMDB Pose TrackingPCK@0.380.8ColorPointer
Activity RecognitionJHMDB Pose TrackingPCK@0.487.5ColorPointer
Activity RecognitionJHMDB Pose TrackingPCK@0.591.4ColorPointer
Action LocalizationJHMDB Pose TrackingPCK@0.145.2ColorPointer
Action LocalizationJHMDB Pose TrackingPCK@0.269.6ColorPointer
Action LocalizationJHMDB Pose TrackingPCK@0.380.8ColorPointer
Action LocalizationJHMDB Pose TrackingPCK@0.487.5ColorPointer
Action LocalizationJHMDB Pose TrackingPCK@0.591.4ColorPointer
Action DetectionJHMDB Pose TrackingPCK@0.145.2ColorPointer
Action DetectionJHMDB Pose TrackingPCK@0.269.6ColorPointer
Action DetectionJHMDB Pose TrackingPCK@0.380.8ColorPointer
Action DetectionJHMDB Pose TrackingPCK@0.487.5ColorPointer
Action DetectionJHMDB Pose TrackingPCK@0.591.4ColorPointer
3D Action RecognitionJHMDB Pose TrackingPCK@0.145.2ColorPointer
3D Action RecognitionJHMDB Pose TrackingPCK@0.269.6ColorPointer
3D Action RecognitionJHMDB Pose TrackingPCK@0.380.8ColorPointer
3D Action RecognitionJHMDB Pose TrackingPCK@0.487.5ColorPointer
3D Action RecognitionJHMDB Pose TrackingPCK@0.591.4ColorPointer
Action RecognitionJHMDB Pose TrackingPCK@0.145.2ColorPointer
Action RecognitionJHMDB Pose TrackingPCK@0.269.6ColorPointer
Action RecognitionJHMDB Pose TrackingPCK@0.380.8ColorPointer
Action RecognitionJHMDB Pose TrackingPCK@0.487.5ColorPointer
Action RecognitionJHMDB Pose TrackingPCK@0.591.4ColorPointer

Related Papers

Channel-wise Motion Features for Efficient Motion Segmentation2025-07-17An Efficient Approach for Muscle Segmentation and 3D Reconstruction Using Keypoint Tracking in MRI Scan2025-07-11What You Have is What You Track: Adaptive and Robust Multimodal Tracking2025-07-08Learning to Track Any Points from Human Motion2025-07-08TLB-VFI: Temporal-Aware Latent Brownian Bridge Diffusion for Video Frame Interpolation2025-07-07Zero-shot Skeleton-based Action Recognition with Prototype-guided Feature Alignment2025-07-01MEMFOF: High-Resolution Training for Memory-Efficient Multi-Frame Optical Flow Estimation2025-06-29R1-Track: Direct Application of MLLMs to Visual Object Tracking via Reinforcement Learning2025-06-27