Research map / Label-efficient and robust learning / Domain adaptation and generalization
Unsupervised Domain Adaptative Temporal Sentence Localization with Mutual Information Maximization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UDA-TSL, transferring temporal sentence localization from a labeled source video-query dataset to an unannotated target dataset, using modality-specific domain adaptation via an intermediate domain plus a mutual-information-based mutual-nearest-neighbor contrastive alignment on target pairs.
Problem
Temporal sentence localization needs expensive annotations and models generalize poorly under domain shift across datasets; the paper targets unsupervised domain adaptation where the target domain has no annotations.
Approach
Modality-specific domain adaptation that aligns video and query distributions through an optimized intermediate domain along the geodesic path between source and target, combined with a mutual-nearest-neighbor video-query matching and contrastive loss for self-supervised target learning.
Key results
- On Charades→ActivityNet, the proposed UDA model reaches R@1 IoU=0.5 of 49.48 versus 44.06 for the best baseline MMN.
- On TACoS→Charades (R@1 IoU=0.5), the UDA model scores 41.39 versus 36.45 for CSMGAN.
- Ablation: removing the MIM contrastive loss drops Charades→ActivityNet R@1 IoU=0.5 from 49.48 to 40.65.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the proposed UDA framework | TSL model · Charades-STA→ActivityNet Captions | R@1, IoU=0.5: 44.06 -> 49.48 | Helped |
| Add modality-specific domain adaptation (ablation) | UDA (Ours) · Charades-STA→ActivityNet Captions | R@1, IoU=0.5: 43.34 -> 49.48 | Helped |
| Add target contrastive learning (MIM) (ablation) | UDA (Ours) · Charades-STA→ActivityNet Captions | R@1, IoU=0.5: 40.65 -> 49.48 | Helped |
| Add domain adaptation and target contrastive learning together (ablation) | UDA (Ours) · Charades-STA→ActivityNet Captions | R@1, IoU=0.5: 35.83 -> 49.48 | Helped |
| Replace direct source-target distribution closing with intermediate-domain adaptation (ablation) | UDA (Ours) · Charades-STA→ActivityNet Captions | R@1, IoU=0.5: 46.99 -> 49.48 | Helped |
| Use reciprocal video-query matching instead of selecting one modality's nearest match (ablation) | MIM module · Charades-STA→ActivityNet Captions | R@1, IoU=0.5: 46.89 -> 49.48 | Helped |
| Use reciprocal video-query matching instead of selecting a unique video match (ablation) | MIM module · Charades-STA→ActivityNet Captions | R@1, IoU=0.5: 46.57 -> 49.48 | Helped |
| Add the proposed UDA framework | TSL model · ActivityNet Captions→TACoS | R@1, IoU=0.5: 22.08 -> 29.83 | Helped |
| Add the proposed UDA framework | TSL model · TACoS→Charades-STA | R@1, IoU=0.5: 33.73 -> 41.39 | Helped |
| Add the proposed UDA framework | TSL model · TACoS→ActivityNet Captions | R@1, IoU=0.5: 20.86 -> 26.16 | Helped |
| Add the proposed UDA framework | TSL model · Charades-STA→TACoS | R@1, IoU=0.5: 19.17 -> 25.48 | Helped |
| Add the proposed UDA framework | TSL model · ActivityNet Captions→Charades-STA | R@1, IoU=0.5: 50.78 -> 60.26 | Helped |
Surprising findings
- Existing TSL models re-implemented under the UDA setting perform worse than their originally reported supervised results, and the no-adaptation baseline NoUDA performs even worse than prior works.
Limitations
- Performance under UDA remains below supervised results reported for these datasets.
- Relies on hand-tuned domain factor alpha and inference thresholds per dataset.
Experimental setup
Datasets: ActivityNet Captions TACoS Charades-STA
Benchmarks: ActivityNet Captions TACoS Charades-STA
Metrics: R@1 IoU=0.5 R@1 IoU=0.7 R@5 IoU=0.5 R@5 IoU=0.7
Baselines: CBP SCDM CMIN CSMGAN 2DTAN DRN MMN NoUDA
Builds on: MMN 2D-TAN CMIN Gopalan geodesic domain adaptation
Closest papers by meaning
- Transferable Video Moment Localization by Moment-Guided Query Prompting (AAAI 2024 · similarity 0.87)
- Omnipotent Distillation with LLMs for Weakly-Supervised Natural Language Video Localization: When Divergence Meets Consistency (AAAI 2024 · similarity 0.87)
- Boundary-Aware Temporal Dynamic Pseudo-Supervision Pairs Generation for Zero-Shot Natural Language Video Localization (AAAI 2025 · similarity 0.87)
- Multi-Pair Temporal Sentence Grounding via Multi-Thread Knowledge Transfer Network (AAAI 2025 · similarity 0.87)
- Weakly Supervised Temporal Action Localization via Dual-Prior Collaborative Learning Guided by Multimodal Large Language Models (CVPR 2025 · similarity 0.87)
- Localizing Events in Videos with Multimodal Queries (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.