Research map / Vision-language and multimodal / Human and embodied modeling
Dense and Aligned Captions (DAC) Promote Compositional Reasoning in VL Models
Sivan Doveh, Assaf Arbelle, Sivan Harary, Roei Herzig, Donghyun Kim, Paola Cascante-Bonilla, Amit Alfassy, Rameswar Panda, Raja Giryes, Rogerio Feris, Shimon Ullman, Leonid Karlinsky
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune CLIP with quality-enhanced captions, LLM-expanded caption bags, negative text augmentation, and negative-enhanced MIL | OpenAI CLIP ViT-B/32 · VL-Checklist + ARO | Avg: 79.42 -> 85.18 | Improved |
| Fine-tune CLIP with quality-enhanced captions, SAM segment captions, negative text augmentation, and negative-enhanced MIL | OpenAI CLIP ViT-B/32 · VL-Checklist Relation | Relation accuracy: 63.05 -> 89.75 | Improved |
| Add caption quality enhancement (ablation) | CLIP fine-tuning with caption expansion · VL-Checklist + ARO | Avg: 61.16% -> 74.38% | Improved |
| Add LLM caption density expansion to quality-enhanced captions with negative augmentation (ablation) | CLIP fine-tuning without MIL · VL-Checklist + ARO | Avg: 60.20% -> 78.09% | Improved |
| Add negative-enhanced MIL instead of randomly choosing one caption from the bag (ablation) | Quality-enhanced CLIP fine-tuning with negative augmentation and LLM expansion · VL-Checklist Relation | Relation accuracy: 81.97% -> 86.41% | Improved |
| Add negative examples to the base MIL loss (ablation) | LLM-expanded CLIP fine-tuning · VL-Checklist Relation | Relation accuracy: 69.05% -> 86.41% | Improved |
| Use MIL aggregation rather than MAX selection from the caption bag (ablation) | LLM-expanded CLIP fine-tuning · VL-Checklist Relation | Relation accuracy: 60.35% -> 69.05% | Improved |
| Use MIL aggregation rather than averaging caption embeddings (ablation) | LLM-expanded CLIP fine-tuning · VL-Checklist Relation | Relation accuracy: 65.57% -> 69.05% | Improved |
| Replace negative-enhanced MIL with MAX selection (ablation) | LLM-expanded CLIP fine-tuning · VL-Checklist Relation | Relation accuracy: 86.41% -> 60.35% | Hurt |
| Replace negative-enhanced MIL with AVG selection (ablation) | LLM-expanded CLIP fine-tuning · VL-Checklist Relation | Relation accuracy: 86.41% -> 65.57% | Hurt |
| Use LLM-based rather than SAM-based caption density expansion (ablation) | DAC fine-tuning · VL-Checklist + ARO | Avg: 85.24% vs 83.83% | Mixed |
| Use SAM-based rather than LLM-based caption density expansion (ablation) | DAC fine-tuning · VL-Checklist + ARO | Avg: 83.83% vs 85.24% | Mixed |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- BLIP2 — BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation (ICML 2022) · better on 1 of 1 shared result
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Contrasting Intra-Modal and Ranking Cross-Modal Hard Negatives to Enhance Visio-Linguistic Compositional Understanding (CVPR 2024 · similarity 0.93)
- Enhancing Compositional Reasoning in CLIP via Reconstruction and Alignment of Text Descriptions (NeurIPS 2025 · similarity 0.92)
- Natural Language Inference Improves Compositionality in Vision-Language Models (ICLR 2025 · similarity 0.92)
- ConMe: Rethinking Evaluation of Compositional Reasoning for Modern VLMs (NeurIPS 2024 · similarity 0.91)
- FINECAPTION: Compositional Image Captioning Focusing on Wherever You Want at Any Granularity (CVPR 2025 · similarity 0.91)
- Not Only Text: Exploring Compositionality of Visual Representations in Vision-Language Models (CVPR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
