Research map / Video / Video segmentation and tracking
Learning Fine-Grained Features for Pixel-Wise Video Correspondences
Rui Li, Shenglong Zhou, Dong Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use soft labels from a pretrained 2D encoder instead of Dirac hard labels (ablation) | synthetic supervision of fine-grained features · TAP-Vid-DAVIS | average position accuracy: 42.6 -> 55.8 | Improved |
| use Gaussian labels instead of Dirac labels, with larger variance (ablation) | synthetic supervision · TAP-Vid-DAVIS | average position accuracy: 42.6 -> 41.9 (sigma 1), 29.7 (sigma 6) | Hurt |
| use a stronger ImageNet contrastive 2D encoder for soft labels (ablation) | soft labeling · TAP-Vid-DAVIS | average position accuracy: 55.8 -> 57.5 | Improved |
| add self-supervised reconstruction loss on real videos to synthetic KL loss (ablation) | fine-grained feature learning · TAP-Vid-DAVIS | average position accuracy: 55.8 -> 59.2 | Improved |
| add adversarial training with gradient reversal layer (ablation) | fine-grained feature learning · TAP-Vid-DAVIS | average position accuracy: 59.2 -> 59.8 | Improved |
| add real-world unlabeled data to reconstruction training (ablation) | L_rec · TAP-Vid-DAVIS | average position accuracy: 56.4 -> 56.7 | Improved |
| use coarse-to-fine matching with stride 8 (ablation) | fine-grained features · TAP-Vid-DAVIS | average position accuracy: 59.8 -> 48.0 (8 FPS -> 34 FPS) | Mixed |
| learn fine-grained features instead of object-centric ones | video object segmentation on DAVIS-17 · DAVIS-17 | J&F_m: 72.1 -> 72.4 (vs LIIR) | Mixed |
Closest papers by meaning
- Semantic-Aware Fine-Grained Correspondence (ECCV 2022 · similarity 0.90)
- Joint-task Self-supervised Learning for Temporal Correspondence (NeurIPS 2019 · similarity 0.88)
- Rethinking Self-Supervised Correspondence Learning: A Video Frame-Level Similarity Perspective (ICCV 2021 · similarity 0.87)
- Spatial-Then-Temporal Self-Supervised Learning for Video Correspondence (CVPR 2023 · similarity 0.86)
- Contrastive Transformation for Self-supervised Correspondence Learning (AAAI 2021 · similarity 0.86)
- Towards Efficient and Effective Text-to-Video Retrieval with Coarse-to-Fine Visual Representation Learning (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
