Research map / Vision-language and multimodal / Multimodal LLMs
Escaping the Likelihood Trap: Geometric Diversity Optimization for Long-Form Image Captioning
Qingmei Tang, Shuai Hao, Rong Fu, Zirui Mo, Xiang Liu, Jiaxuan Lu, Wenyu Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Identifies a "Likelihood Trap" in which MLE and KL-regularized alignment push VLM captions toward generic high-probability templates, and proposes a geometric diversity optimization approach to recover fine-grained detail in long-form image captions.
Problem
VLMs produce generic, template-like long-form image captions because standard alignment objectives (MLE and KL regularization) favor high-probability outputs and suppress fine-grained visual details.
Approach
Frames the caption-generation collapse as a likelihood trap caused by MLE and KL-regularization, and counters it with a geometric diversity optimization objective rather than likelihood-based alignment alone.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add DPP volume reward (ablation) | Geo-RL · DeCapBench | Distinct-2: 0.72 -> 0.84 | Improved |
| add DPP volume reward (ablation) | Geo-RL · DeCapBench | SVS: 2.65 -> 3.55 | Improved |
| add DPP volume reward (ablation) | Geo-RL · DeCapBench | CHAIRi: 7.7 -> 7.1 | Improved |
| add contrastive repulsion (ablation) | Geo-RL · DeCapBench | Distinct-2: 0.77 -> 0.84 | Improved |
| add contrastive repulsion (ablation) | Geo-RL · DeCapBench | SVS: 2.95 -> 3.55 | Improved |
| use RBF kernel instead of linear kernel (ablation) | Geo-RL DPP reward · DeCapBench | SVS: 2.85 -> 3.55 | Improved |
| use RBF kernel instead of polynomial (deg=2) kernel (ablation) | Geo-RL DPP reward · DeCapBench | Distinct-2: 0.68 -> 0.84 | Improved |
| use Sentence-BERT instead of SimCSE encoder (ablation) | Geo-RL · DeCapBench | SVS: 3.21 -> 3.55 | Improved |
| use 8 samples per image instead of 4 (ablation) | Geo-RL · DeCapBench | SVS: 3.05 -> 3.55 | Improved |
| use 8 samples per image instead of 32 (ablation) | Geo-RL · DeCapBench | SVS: 3.55 -> 3.32 | Hurt |
| train on 50K images instead of 10K (ablation) | Geo-RL · DeCapBench | SVS: 3.01 -> 3.55 | Improved |
| train on 100K images instead of 50K (ablation) | Geo-RL · DeCapBench | SVS: 3.55 -> 3.50 | No clear effect |
Surprising findings
- Standard MLE and KL-regularized alignment suppress fine-grained detail in VLM captions, producing generic high-probability templates (the Likelihood Trap).
Experimental setup
Builds on: MLE training KL-regularized alignment
Closest papers by meaning
- Top-Down Semantic Refinement for Image Captioning (AAAI 2026 · similarity 0.88)
- BalCapRL : A Balanced Framework for RL-Based MLLM Image Captioning (NeurIPS 2026 · similarity 0.88)
- RegionGPT: Towards Region Understanding Vision Language Model (CVPR 2024 · similarity 0.87)
- GeoDiT: A Diffusion-based Vision-Language Model for Geospatial Understanding (CVPR 2026 · similarity 0.87)
- Painting with Words: Elevating Detailed Image Captioning with Benchmark and Alignment Learning (ICLR 2025 · similarity 0.86)
- Benchmarking Large Vision-Language Models via Directed Scene Graph for Comprehensive Image Captioning (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
