Research map / Vision-language and multimodal / Multimodal LLMs
Connecting Vision and Language with Localized Narratives
Jordi Pont-Tuset, Jasper Uijlings, Soravit Changpinyo, Radu Soricut, Vittorio Ferrari
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use spoken narration with mouse traces instead of two-stage entity marking and box drawing | Localized Narratives annotation · Annotation cost | seconds per image: 643 seconds per image for Flickr30k Entities, 4.4x slower | Improved |
| add automatic quality control matching spoken and written transcripts | Localized Narratives annotation · Annotation quality | fraction of annotations discarded: 23.5% of all annotations discarded | Improved |
| describe an image with a paragraph rather than one sentence | Localized Narratives vs COCO Captions · Caption richness | nouns per caption: 3.6 -> 10.8 | Improved |
| add per-word mouse trace grounding to paragraph captions | Localized Narratives vs Stanford Visual Paragraphs · Dataset scale | images annotated: 40x more images than Stanford Visual Paragraphs | Improved |
| use free-form mouse hover traces | Localized Narratives · Localization accuracy | trace points inside correct box | Mixed |
Closest papers by meaning
- HoVLE: Unleashing the Power of Monolithic Vision-Language Models with Holistic Vision-Language Embedding (CVPR 2025 · similarity 0.85)
- Natural Language Inference Improves Compositionality in Vision-Language Models (ICLR 2025 · similarity 0.84)
- From Pixels to Words -- Towards Native Vision-Language Primitives at Scale (ICLR 2026 · similarity 0.84)
- Aligning Forest and Trees in Images & Long Captions for Visually Grounded Understanding (NeurIPS 2026 · similarity 0.83)
- JourneyBench: A Challenging One-Stop Vision-Language Understanding Benchmark of Generated Images (NeurIPS 2024 · similarity 0.83)
- Beyond Bag-of-Words: Diagnosing Compositional Binding Failures in Vision-Language Models (NeurIPS 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
