Research map / Video / Video-language understanding
Ambiguity-Restrained Text-Video Representation Learning for Partially Relevant Video Retrieval
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Ambiguity-Restrained representation Learning (ARL) for partially relevant video retrieval, treating uncertain, highly similar unpaired text-video and text-frame pairs as ambiguous rather than negative, with cross-model ambiguity detection.
Problem
Partially relevant video retrieval is trained with one-to-one text-video labels, so unpaired but relevant videos are wrongly treated as negatives, creating label ambiguity.
Approach
Detect ambiguous pairs using dataset-level uncertainty (average similarity across the dataset) combined with pair-wise similarity, then relax their constraints via multi-positive contrastive loss and dual triplet margin losses; extend this to text-frame level and use two co-trained models to detect ambiguity for each other.
Key results
- On TVR with ResNet/I3D/RoBERTa features, achieves SumR 185.9 vs GMMFormer 176.6 (+9.3 points).
- With CLIP-L/14 features, SumR 260.1 on TVR vs MS-SL 251.0 and GMMFormer 241.1; 190.4 on ActivityNet vs 187.1 for GMMFormer.
- Ablation on TVR: text-video ambiguity learning raises SumR from 252.8 to 256.4; adding text-frame ambiguity reaches 258.7 and full model 260.1.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ARL instead of GMMFormer | PRVR retrieval with ResNet, I3D, and RoBERTa features · TVR | SumR: 176.6 -> 185.9 | Helped |
| use ARL instead of DL-DKD | PRVR retrieval with ResNet, I3D, and RoBERTa features · ActivityNet Captions | SumR: 147.6 -> 148.3 | Helped |
| use ARL instead of MS-SL | PRVR retrieval with CLIP-L/14 features · TVR | SumR: 251.0 -> 260.1 | Helped |
| use ARL instead of GMMFormer | PRVR retrieval with CLIP-L/14 features · ActivityNet Captions | SumR: 187.1 -> 190.4 | Helped |
| add text-video ambiguity-aware representation learning (ablation) | ARL base model · TVR | SumR: 252.8 -> 256.4 | Helped |
| add text-frame ambiguity learning (ablation) | ARL with text-video ambiguity learning · TVR | SumR: 256.4 -> 258.7 | Helped |
| add cross-model learning (ablation) | ARL with text-video ambiguity learning · TVR | SumR: 256.4 -> 258.4 | Helped |
| add cross-model learning alongside text-frame ambiguity learning (ablation) | ARL with text-video ambiguity learning · TVR | SumR: 258.7 -> 260.1 | Helped |
| treat all ambiguous pairs as positives instead of using the proposed ambiguous-set strategy (ablation) | ARL · TVR | SumR: 260.1 -> 258.3 | Hurt |
| ignore ambiguous pairs instead of using the proposed ambiguous-set strategy (ablation) | ARL · TVR | SumR: 260.1 -> 259.2 | Hurt |
Surprising findings
- Treating ambiguous sets as positives or ignoring them both lowered performance relative to the flexible multi-positive treatment.
- Dataset-level uncertainty distributions are similar for positive and unpaired pairs, while similarity separates them.
Limitations
- Runtime is slower than GMMFormer despite fewer FLOPs.
- Thresholds are set heuristically from distribution means rather than tuned in a principled way.
Experimental setup
Datasets: TVR ActivityNet Captions
Benchmarks: TVR ActivityNet Captions
Metrics: R@1 R@5 R@10 R@100 SumR
Baselines: XML ReLoCLNet CONQUER MS-SL DL-DKD GMMFormer
Builds on: MS-SL GMMFormer supervised contrastive learning DL-DKD
Closest papers by meaning
- Mitigating Semantic Collapse in Partially Relevant Video Retrieval (NeurIPS 2025 · similarity 0.91)
- Bridging the Semantic Granularity Gap Between Text and Frame Representations for Partially Relevant Video Retrieval (AAAI 2025 · similarity 0.90)
- Diffusion-Inspired Truncated Sampler for Text-Video Retrieval (NeurIPS 2024 · similarity 0.87)
- Bridging Information Asymmetry in Text-video Retrieval: A Data-centric Approach (ICLR 2025 · similarity 0.87)
- Rebalancing Contrastive Alignment with Bottlenecked Semantic Increments in Text-Video Retrieval (NeurIPS 2025 · similarity 0.86)
- Learning Dynamic Similarity by Bidirectional Hierarchical Sliding Semantic Probe for Efficient Text Video Retrieval (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.