Research map / Vision-language and multimodal / Multimodal LLMs
Beyond Final Answers: CRYSTAL Benchmark for Transparent Multimodal Reasoning Evaluation
Wayner Barrios, SouYoung Jin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train with CPR-Curriculum via GRPO | Qwen2.5-VL-3B · CRYSTAL test | Match F1: 0.480 -> 0.633 | Improved |
| train with CPR-Curriculum via GRPO | Qwen2.5-VL-3B · CRYSTAL test | accuracy (%): 39.85 -> 47.52 | Improved |
| train with CPR-Curriculum via GRPO | InternVL3.5-4B · CRYSTAL test | Match F1: 0.432 -> 0.833 | Improved |
| train with CPR-Curriculum via GRPO | InternVL3.5-4B · CRYSTAL test | accuracy (%): 37.61 -> 45.76 | Improved |
| train with CPR-Curriculum via GRPO | Qwen2.5-VL-3B · CRYSTAL test | LIS: 0.723 -> 0.626 | Hurt |
| replace additive Composite reward with multiplicative CPR (ablation) | Qwen2.5-VL-3B GRPO · CRYSTAL test | Match F1: 0.426 -> 0.633 | Improved |
| replace Answer-Only reward with multiplicative CPR (ablation) | Qwen2.5-VL-3B GRPO · CRYSTAL test | Match F1: 0.429 -> 0.633 | Improved |
| replace Composite reward with CPR (ablation) | Qwen2.5-VL-3B GRPO · CRYSTAL test | accuracy (%): 44.92 -> 41.40 | Hurt |
| add curriculum (answer-only warm-up, difficulty schedule) to CPR (ablation) | Qwen2.5-VL-3B GRPO · CRYSTAL test | accuracy (%): 41.40 -> 47.52 | Improved |
| add curriculum to CPR (ablation) | Qwen2.5-VL-3B GRPO · CRYSTAL test | Match F1: 0.633 -> 0.633 | No clear effect |
| set reward weights a_w=0.65, s_w=0.35 instead of a_w=0.80, s_w=0.20 (ablation) | CPR · held-out val | accuracy (%): 81.79 -> 82.70 | Improved |
| set reward weights a_w=0.65, s_w=0.35 instead of a_w=0.20, s_w=0.80 (ablation) | CPR · held-out val | accuracy (%): 74.81 -> 82.70 | Improved |
Closest papers by meaning
- RPTS: Tree-Structured Reasoning Process Scoring for Faithful Multimodal Evaluation (AAAI 2026 · similarity 0.92)
- LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models (ICLR 2026 · similarity 0.92)
- MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiency (ICML 2025 · similarity 0.91)
- Unveiling Fine-Grained Visual Traces: Evaluating MultiModal Interleaved Reasoning Chains in Multimodal STEM Tasks (NeurIPS 2026 · similarity 0.91)
- AStar: Boosting Multimodal Reasoning with Automated Structured Thinking (AAAI 2026 · similarity 0.91)
- MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models (AAAI 2026 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
