Research map / Vision-language and multimodal / Multimodal LLMs
Embodied Image Captioning: Self-supervised Learning Agents for Spatially Coherent Image Descriptions
Tommaso Galliena, Tommaso Apicella, Stefano Rosa, Pietro Morerio, Alessio Del Bue, Lorenzo Natale
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use LD-CPS instead of ECO pseudo-captions | CoCa, CLA · Gibson test | CIDEr: 1.18 -> 1.54 | Improved |
| use LD-CPS instead of IC3 pseudo-captions | CoCa, CLA · Gibson test | CS: 60.34 -> 75.01 | Improved |
| use CLA exploration policy instead of Random with pseudo-captions | LD-CPS CoCa · Gibson test | CS: 71.71 -> 75.01 | Improved |
| add LD-CPS fine-tuning to CoCa (standard) | CoCa · Gibson test, Random | CS: 63.30 -> 70.11 | Improved |
| add triplet loss to fine-tuning (ablation) | CoCa · Gibson test, Random | CS: 70.11 -> 73.26 | Improved |
| add triplet loss to fine-tuning (ablation) | CoCa · Gibson test, Frontier | CIDEr: 1.08 -> 0.94 | Hurt |
| add triplet loss to fine-tuning (ablation) | CoCa · HM3D test, Frontier | CS: 72.47 -> 73.99 | Improved |
| add triplet loss to fine-tuning (ablation) | BLIP2 · HM3D test, Random | SPICE: 27.18 -> 32.15 | Improved |
| add triplet loss to fine-tuning (ablation) | CoCa · HM3D test, Random | B4: 17.34 -> 16.57 | Hurt |
| raise triplet weight from 0.1 to 1.0 (ablation) | CoCa · Gibson test, Frontier | CS: 73.44 -> 69.28 | Hurt |
| use Mistral-7B instead of LLaMa-3 for pseudo-captions (ablation) | LD-CPS BLIP2 · HM3D CLA | SPICE: -1.85 p.p. in SP and -0.42 p.p. in CI | Hurt |
| use Qwen3-8B instead of LLaMa-3 for pseudo-captions (ablation) | LD-CPS BLIP2 · HM3D CLA | SPICE: -1.41 p.p. in SP and -0.13 p.p. in CI | Hurt |
Closest papers by meaning
- MM-SeR: Multimodal Self-Refinement for Lightweight Image Captioning (CVPR 2026 · similarity 0.89)
- VDC-Agent: When Video Detailed Captioners Evolve Themselves via Agentic Self-Reflection (ECCV 2026 · similarity 0.89)
- From Show to Tell: A Survey on Deep Learning-Based Image Captioning (TPAMI 2022 · similarity 0.87)
- Show, Tell and Discriminate: Image Captioning by Self-retrieval with Partially Labeled Data (ECCV 2018 · similarity 0.87)
- CaptionQA: Is Your Caption as Useful as the Image Itself? (CVPR 2026 · similarity 0.86)
- Top-Down Semantic Refinement for Image Captioning (AAAI 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
