Research map / Video / Video generation
I'm a Map! Interpretable Motion-Attentive Maps: Spatio-Temporally Localizing Concepts in Video Diffusion Transformers
Youngjun Jun, Seil Kang, Woojung Han, Seong Jae Hwang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use IMAP instead of ConceptAttention | CogVideoX-5B · MeViS motion localization | MLS average: 0.45 -> 0.62 | Improved |
| use GramCol instead of ConceptAttention | CogVideoX-5B · VSPW zero-shot video semantic segmentation | mIoU: 25.0 -> 28.9 | Improved |
| use GramCol instead of ConceptAttention | CogVideoX-5B · VSPW zero-shot video semantic segmentation | mVC8: 80.4 -> 75.2 | Hurt |
| add AnyUp upsampling to saliency maps | GramCol · VSPW zero-shot video semantic segmentation | mIoU: 28.9 -> 30.1 | Improved |
| replace ConceptAttention with GramCol (no layer or head selection) (ablation) | CogVideoX-5B · MeViS motion localization | MLS average: 0.37 -> 0.41 | Improved |
| add lambda2-based layer selection (ablation) | GramCol · MeViS motion localization | MLS average: 0.41 -> 0.46 | Improved |
| add motion head selection (CHI top-5) (ablation) | GramCol · MeViS motion localization | MLS average: 0.41 -> 0.47 | Improved |
| combine layer selection and motion head selection (ablation) | GramCol · MeViS motion localization | MLS average: 0.41 -> 0.62 | Improved |
| apply softmax over concepts to IMAP (ablation) | IMAP · MeViS motion localization | MLS average: 0.62 -> 0.60 | Mixed |
| select heads with higher Calinski-Harabasz separation score (ablation) | GramCol attention heads · MeViS subset, CogVideoX-2B | MLS: Pearson r = 0.60 | Improved |
Closest papers by meaning
- Emergent Temporal Correspondences from Video Diffusion Transformers (NeurIPS 2025 · similarity 0.88)
- ConceptAttention: Diffusion Transformers Learn Highly Interpretable Features (ICML 2025 · similarity 0.88)
- FRESCO: Spatial-Temporal Correspondence for Zero-Shot Video Translation (CVPR 2024 · similarity 0.88)
- Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfer (CVPR 2024 · similarity 0.88)
- Video Diffusion Models are Training-free Motion Interpreter and Controller (NeurIPS 2024 · similarity 0.87)
- A Motion-aware Spatio-temporal Graph for Video Salient Object Ranking (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
