Research map / Large language models / Language, safety and interpretability
LLM-Free Image Captioning Evaluation in Reference-Flexible Settings
Shinnosuke Hirano, Yuiga Wada, Kazuki Matsuda, Seitaro Otsuki, Komei Sugiura
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add ARSM (learnable similarity) instead of none (ablation) | Pearl · Composite (ref-based) | Kendall tau-c: 30.9 -> 60.4 | Improved |
| use adaptive ARSM instead of non-learnable RUSE-type similarity (ablation) | Pearl · Nebula (ref-based) | Kendall tau-c: 50.3 -> 55.4 | Improved |
| add Stella text encoder (ablation) | Pearl · Composite (ref-based) | Kendall tau-c: 51.9 -> 60.4 | Improved |
| add CLIP encoder (ablation) | Pearl · Composite (ref-based) | Kendall tau-c: 55.5 -> 60.4 | Improved |
| train Pearl on Spica instead of Nebula (ablation) | Pearl · Composite (ref-based) | Kendall tau-c: 59.2 -> 60.4 | Improved |
| train DENEB on Spica instead of Nebula (ablation) | DENEB · Composite (ref-based) | Kendall tau-c: 57.9 -> 58.6 | Improved |
| freeze ARSM at initial values (ablation) | Pearl · Flickr8K-CF (ref-based) | Kendall tau-b: 38.5 vs 38.6 | No clear effect |
| use Pearl instead of previous best LLM-free reference-free metric on Composite | Pearl · Composite (ref-free) | Kendall tau-c: 61.5 -> 58.4 (BLIP2Score vs Pearl) | Hurt |
| use Pearl instead of Ref-HICEScore | Pearl · Flickr8K-Expert (ref-based) | Kendall tau-c: 57.7 -> 58.6 | Improved |
Closest papers by meaning
- CAPability: A Comprehensive Visual Caption Benchmark for Evaluating Both Correctness and Thoroughness (NeurIPS 2025 · similarity 0.89)
- CaptionQA: Is Your Caption as Useful as the Image Itself? (CVPR 2026 · similarity 0.89)
- A Picture is Worth More Than 77 Text Tokens: Evaluating CLIP-Style Models on Dense Captions (CVPR 2024 · similarity 0.89)
- G-VEval: A Versatile Metric for Evaluating Image and Video Captions Using GPT-4o (AAAI 2025 · similarity 0.88)
- Polos: Multimodal Metric Learning from Human Feedback for Image Captioning (CVPR 2024 · similarity 0.88)
- Painting with Words: Elevating Detailed Image Captioning with Benchmark and Alignment Learning (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
