Research map / Detection and segmentation / Open-set and few-shot detection
Compositional Prompt Tuning with Motion Cues for Open-vocabulary Video Relation Detection
Kaifeng Gao, Long Chen, Hanwang Zhang, Jun Xiao, Qianru Sun
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add compositional prompt (separate subject and object contexts) (ablation) | RePro relation prompt · VidVRD-test Novel-split | PredCls R@100: 27.43 -> 30.90 | Improved |
| add motion-cue-based prompt groups (over compositional only) (ablation) | RePro relation prompt · VidVRD-test Novel-split | PredCls R@100: 30.90 -> 33.88 | Improved |
| add motion-cue-based prompt groups (over compositional only) (ablation) | RePro relation prompt · VidVRD-test All-splits | PredCls mAP: 29.45 -> 34.90 | Improved |
| replace motion-based groups with random group selection at train and test (ablation) | RePro relation prompt · VidVRD-test Novel-split | SGDet mAP: 7.14 (Rand) vs 6.10 (full) | Mixed |
| ensemble motion prompts by averaging instead of motion-based selection (ablation) | RePro relation prompt · VidVRD-test All-splits | SGDet mAP: 20.15 (Ens) vs 21.33 (full) | Hurt |
| add compositional prompt (ablation) | RePro relation prompt · VidVRD-test All-splits | SGDet mAP: 19.73 -> 18.47 (compositional without motion) | No clear effect |
| add compositional prompt (ablation) | RePro relation prompt · VidOR-val Novel-split | SGCls R@50: 0.86 -> 1.72 | Improved |
| add motion-cue prompt groups on top of compositional prompts (ablation) | RePro relation prompt · VidOR-val Novel-split | PredCls R@50: 6.62 -> 7.20 (small) | No clear effect |
| full RePro vs fixed-prompt VidVRD-II (re-implemented with ALPro text encoder) | relation classification with learned prompts · VidVRD-test Novel-split | SGDet mAP: 3.57 -> 6.10 | Improved |
| distill from pretrained visual encoder into V2L projection jointly with prompt learning (RePro†) vs RePro (ablation) | relation classification · VidVRD-test Novel-split | SGDet mAP: 2.56 (RePro†) vs 6.10 (RePro) | Hurt |
| add visual distillation (l1 to VLM visual embeddings) to negative-tracklet-loss tracklet detector (ablation) | open-vocab tracklet detection · VidOR-val | R@10: 12.37 -> 16.85 (with BG-Embd) | Improved |
| replace background embedding with negative-tracklet loss (no BG-Embd) (ablation) | open-vocab tracklet detection · VidOR-val | R@10: 16.85 -> 37.08 | Improved |
Closest papers by meaning
- Multi-Modal Prompting for Open-Vocabulary Video Visual Relationship Detection (AAAI 2024 · similarity 0.91)
- Prompt Tuning In a Compact Attribute Space (AAAI 2025 · similarity 0.88)
- OpenworldAUC: Towards Unified Evaluation and Optimization for Open-world Prompt Tuning (ICML 2025 · similarity 0.87)
- Learning to Learn Better Visual Prompts (AAAI 2024 · similarity 0.87)
- Learning Hierarchical Prompt with Structured Linguistic Knowledge for Vision-Language Models (AAAI 2024 · similarity 0.86)
- Relation-aware Hierarchical Prompt for Open-vocabulary Scene Graph Generation (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
