Research map / Vision-language and multimodal / Multimodal LLMs
CompCap: Improving Multimodal Large Language Models with Composite Captions
Xiaohui Chen, Satya Narayan Shukla, Mahmoud Azab, Aashu Singh, Qifan Wang, David Yang, ShengYun Peng, Hanchao Yu, Shen Yan, Xuewen Zhang, Baosheng He
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add CompCap-118K captions to the SFT data | xGen-MM-inst.-4B* · 11 MLLM benchmarks | average score: 57.2 -> 58.9 | Improved |
| add CompCap-118K captions to the SFT data | LLaVA-NeXT-Vicuna-7B · 11 MLLM benchmarks | average score: 62.5 -> 64.5 | Improved |
| add CompCap-118K captions to the SFT data | LLaVA-NeXT-Vicuna-13B · 11 MLLM benchmarks | average score: 65.6 -> 68.5 | Improved |
| add collage captions (ablation) | LLaVA-NeXT-Vicuna-13B · 11 MLLM benchmarks | average score: 65.6 -> 66.4 | Improved |
| add code captions (cumulative) (ablation) | LLaVA-NeXT-Vicuna-13B · 11 MLLM benchmarks | average score: 66.4 -> 66.6 | Improved |
| add table captions (cumulative) (ablation) | LLaVA-NeXT-Vicuna-13B · 11 MLLM benchmarks | average score: 66.6 -> 67.0 | Improved |
| add diagram captions (cumulative) (ablation) | LLaVA-NeXT-Vicuna-13B · 11 MLLM benchmarks | average score: 67.0 -> 67.4 | Improved |
| add chart captions (cumulative) (ablation) | LLaVA-NeXT-Vicuna-13B · 11 MLLM benchmarks | average score: 67.4 -> 68.0 | Improved |
| add image-text captions (cumulative, full CompCap-118K) (ablation) | LLaVA-NeXT-Vicuna-13B · 11 MLLM benchmarks | average score: 68.0 -> 68.5 | Improved |
| add CompCap-118K captions | LLaVA-NeXT-Vicuna-13B · CI-dominated benchmarks | average score: 61.3 -> 64.6 | Improved |
| add CompCap-118K captions | LLaVA-NeXT-Vicuna-13B · NI-dominated benchmarks | average score: 70.9 -> 73.1 | Improved |
| replace ChartQA instruction data with captions (ratio k from 0 to 100 percent) (ablation) | LLaVA-NeXT-Vicuna-13B · ChartQA | accuracy | Mixed |
Closest papers by meaning
- Patch Matters: Training-free Fine-grained Image Caption Enhancement via Local Perception (CVPR 2025 · similarity 0.85)
- CAPability: A Comprehensive Visual Caption Benchmark for Evaluating Both Correctness and Thoroughness (NeurIPS 2025 · similarity 0.85)
- MM-SeR: Multimodal Self-Refinement for Lightweight Image Captioning (CVPR 2026 · similarity 0.84)
- CapsFusion: Rethinking Image-Text Data at Scale (CVPR 2024 · similarity 0.84)
- Revisit Large-Scale Image-Caption Data in Pre-training Multimodal Foundation Models (ICLR 2025 · similarity 0.84)
- A Picture is Worth More Than 77 Text Tokens: Evaluating CLIP-Style Models on Dense Captions (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
