Research map / Vision-language and multimodal / Human and embodied modeling
Where Do Long Captions Fail? Position-Aware Diagnosis and Reinforcement Learning for Detailed Image Captioning
Yuanze Hu, Zhichao Yang, Junwei Jing, Xin Yang, Wenxuan Liu, Tinghai Zhang, Jia Fu, Hongzhi Zhang, Xiangyu Wu, Ruiming Tang, Tingting Gao, Han Li
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Diagnoses how detailed image captions fail across sentence positions using a position-aware rubric of atomic visual claims, and uses this to guide reinforcement learning for long-form captioning.
Problem
Detailed image captioning is usually framed as trading off coverage of visual facts against hallucination, which hides that captions are ordered sequences where different failures occur at different positions.
Approach
Decomposes dense human reference captions into atomic visual claims arranged by position in the caption, enabling position-aware diagnosis of failures and a position-aware reinforcement learning signal for long captions.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add middle-position reward for supported fine-grained evidence (R_mid) (ablation) | PAGER · diagnostic subset | D_mid: +.103 -> +.036 | Improved |
| add tail generic-escape penalty (R_tail) (ablation) | PAGER · diagnostic subset | S_Q4: .374 -> .436 | Improved |
| add tail generic-escape penalty (R_tail) (ablation) | PAGER · diagnostic subset | G_Q4: .378 -> .329 | Improved |
| add content-aware length budget (R_len) (ablation) | PAGER · diagnostic subset | D_mid: +.047 -> +.036 | Improved |
| add PixMo rubric filtering (ablation) | PAGER · diagnostic subset | D_mid: +.064 -> +.036 | Improved |
| use itemwise evidence-span reward instead of holistic checklist (ablation) | PAGER · diagnostic subset | D_mid: +.058 -> +.036 | Improved |
| split middle/tail rewards instead of a monotone late-position weight (ablation) | PAGER · diagnostic subset | H_Q4: .291 -> .235 | Improved |
| monotone late-position weight vs split rewards on D_mid (ablation) | PAGER · diagnostic subset | D_mid: +.037 vs +.036 | No clear effect |
| apply PAGER GRPO to Qwen3-VL-4B-Instruct | Qwen3-VL-4B · diagnostic subset | D_mid: +.096 -> +.036 | Improved |
| apply PAGER GRPO | Qwen3-VL-4B · diagnostic subset | H_Q4: .298 -> .235 | Improved |
| PAGER versus CapRL on DCScore-F1 | PAGER · IIW-400 | DCScore-F1: .552 (CapRL) vs .526 | Hurt |
| PAGER versus CapRL on tail genericity | PAGER · diagnostic subset | D_tail: +.123 (CapRL) vs +.177 | Hurt |
Experimental setup
Datasets: dense human reference captions
Closest papers by meaning
- CAPability: A Comprehensive Visual Caption Benchmark for Evaluating Both Correctness and Thoroughness (NeurIPS 2025 · similarity 0.89)
- Painting with Words: Elevating Detailed Image Captioning with Benchmark and Alignment Learning (ICLR 2025 · similarity 0.89)
- CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning (ICLR 2026 · similarity 0.88)
- CaptionQA: Is Your Caption as Useful as the Image Itself? (CVPR 2026 · similarity 0.88)
- Top-Down Semantic Refinement for Image Captioning (AAAI 2026 · similarity 0.88)
- Toward Robust Hyper-Detailed Image Captioning: A Multiagent Approach and Dual Evaluation Metrics for Factuality and Coverage (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
