Research map / Vision-language and multimodal / Human and embodied modeling
Seeing What’s Wrong: A Trajectory-Guided Approach to Caption Error Detection
Gabriel Afriat, Ryan Lucas, Xiang Meng, Yufang Hou, Yada Zhu, Rahul Mazumder
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add TRACED caption trajectory on top of baseline scorer | BLIP (ITM) · Flickr30k error detection | accuracy: 88.5 -> 89.5 | Improved |
| add TRACED on top of CLIP | CLIP · Flickr30k error detection | accuracy: 83.8 -> 85.7 | Improved |
| use Elimination instead of GCD/Fast GCD for trajectories (ablation) | TRACED · MM-IMDb error detection | accuracy: LEMON FIX 77.6 (GCD) -> 78.3 (Elim) | Improved |
| use the full trajectory instead of only the first step (ablation) | TRACED · Flickr30k error detection | % change in accuracy: -1.25 for first step only | Improved |
| use full trajectory instead of last step only (ablation) | TRACED · Flickr30k error detection | % change in accuracy: -43.07 for last step only | Improved |
| use full trajectory instead of mean trajectory (ablation) | TRACED · Flickr30k error detection | % change in accuracy: -4.90 for mean only | Improved |
| combine alignment s and similarity c instead of either alone (ablation) | TRACED · MS-COCO error detection | % change in accuracy: -0.41 (s only), -6.03 (c only) | Improved |
| maximize instead of minimize the alignment score when building trajectories (ablation) | TRACED · Flickr30k error detection | % change in accuracy: -0.76 for minimizing (BLIP ITM) | Improved |
| increase trajectory length T (ablation) | TRACED · error detection | accuracy | Improved |
| filter with TRACED-BLIP instead of BLIP before fine-tuning | BLIP-2 · Flickr30k captioning | BLEU-4: 30.8 -> 31.8 | Improved |
| filter with BLIP baseline instead of keeping noisy data | BLIP-2 · Flickr30k captioning | BLEU-4: 31.5 -> 30.8 | Hurt |
| add TRACED hints to InternVL3-1B correction (ablation) | InternVL3-1B · Flickr30k captioning | BLEU-4: 32.0 -> 32.5 | Improved |
Closest papers by meaning
- LEMoN: Label Error Detection using Multimodal Neighbors (ICML 2025 · similarity 0.88)
- Symbal: Detecting Systematic Misalignments in Model-Generated Captions (ICML 2026 · similarity 0.86)
- CAPability: A Comprehensive Visual Caption Benchmark for Evaluating Both Correctness and Thoroughness (NeurIPS 2025 · similarity 0.85)
- Noise-Aware Image Captioning with Progressively Exploring Mismatched Words (AAAI 2024 · similarity 0.84)
- Extract Free Dense Misalignment from CLIP (AAAI 2025 · similarity 0.84)
- CaptionQA: Is Your Caption as Useful as the Image Itself? (CVPR 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
