Research map / Vision-language and multimodal / Human and embodied modeling
Personalized Image Descriptions from Attention Sequences
Ruoyu Xue, Hieu Le, Jingyi Xu, Sounak Mondal, Abe Leite, Gregory Zelinsky, Minh Hoai, Dimitris Samaras
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add attention-trajectory input to DEPER (ablation) | DEPER · Flickr30k-LN (seen, ablation) | CLS accuracy: 0.649 -> 0.796 | Improved |
| add attention-trajectory input to DEPER (ablation) | DEPER · Flickr30k-LN (seen, ablation) | CIDEr: 0.770 -> 0.789 | No clear effect |
| add trajectory dynamics (duration, order) (ablation) | DEPER · Flickr30k-LN (seen, ablation) | BLEU-4: 0.276 -> 0.312 | Improved |
| add trajectory reconstruction objective (ablation) | DEPER · Flickr30k-LN (seen, ablation) | BLEU-4: 0.230 -> 0.312 | Improved |
| add dual-context encoder (ablation) | DEPER · Flickr30k-LN (seen, ablation) | CIDEr: 0.729 -> 0.789 | Improved |
| add contrastive loss (ablation) | DEPER · Flickr30k-LN (seen, ablation) | CLS accuracy: 0.722 -> 0.796 | Improved |
| add FiLM modulation in trajectory decoder (ablation) | DEPER · Flickr30k-LN (seen, ablation) | CIDEr: 0.723 -> 0.789 | Improved |
| add trajectory latent to decoder (ablation) | DEPER · Flickr30k-LN (seen, ablation) | BLEU-4: 0.272 -> 0.312 | Improved |
| use full 801 samples per subject vs 100 (ablation) | DEPER · Flickr30k-LN (seen, ablation) | CIDEr: 0.400 -> 0.789 | Improved |
| replace prompt tuning with DEPER embeddings | Qwen2-VL-2B · COCO-LN (seen) | CIDEr: 0.587 -> 0.726 | Improved |
| replace prompt tuning with DEPER embeddings | Qwen2-VL-2B · Kollenda et al. (seen) | CLS accuracy: 0.056 -> 0.083 | Improved |
Closest papers by meaning
- Ego: Embedding-Guided Personalization of Vision-Language Models (CVPR 2026 · similarity 0.87)
- Selectively Informative Description can Reduce Undesired Embedding Entanglements in Text-to-Image Personalization (CVPR 2024 · similarity 0.87)
- Yo'Chameleon: Personalized Vision and Language Generation (CVPR 2025 · similarity 0.86)
- What's in the Image? A Deep-Dive into the Vision of Vision Language Models (CVPR 2025 · similarity 0.86)
- Contextualized Visual Personalization in Vision-Language Models (ICML 2026 · similarity 0.86)
- The Many Senses of Visual Similarity: A Text-Prompted Image Perceptual Metric (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
