Research map / Vision-language and multimodal / Multimodal LLMs
MINT-CoT: Enabling Interleaved Visual Tokens in Mathematical Chain-of-Thought Reasoning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MINT-CoT, which uses a learned Interleave Token to select arbitrary-shaped visual tokens from math figures and interleave them into each reasoning step, trained via text CoT SFT, interleaved SFT, and GRPO-based RL on a 54K token-level dataset, yielding MINT-CoT-7B.
Problem
Multimodal mathematical chain-of-thought reasoning is weak because MLLMs perceive math figures poorly, box-shaped visual interleaving is too coarse, and existing visual-CoT methods depend on external tools or generation.
Approach
An Interleave Token generated before each reasoning step is compared via scaled cosine similarity with projected visual token states; tokens above a threshold are selected and inserted into the reasoning, enabling token-level, arbitrary-shape visual grounding without external tools, supervised by a 54K token-indexed dataset built with grid cells, OCR, and GPT-4o annotation.
Key results
- MINT-CoT-7B reaches 73.70 on MathVista-Math, +32.59 over the Qwen2-VL-7B baseline (41.11).
- MINT-CoT-7B reaches 64.72 on GeoQA vs 37.80 for the baseline (+26.92).
- MINT-CoT-7B reaches 69.6 on MMStar-Math vs 46.4 for the baseline (+23.2).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train with text-only CoT SFT (ablation) | Qwen2-VL-7B-Instruct · MMStar-Math | accuracy: 46.4 -> 67.6 | Helped |
| Train with text-only CoT SFT (ablation) | Qwen2-VL-7B-Instruct · GeoQA | accuracy: 37.80 -> 59.02 | Helped |
| Train with text-only CoT SFT (ablation) | Qwen2-VL-7B-Instruct · MathVista-Math | accuracy: 41.11 -> 64.07 | Helped |
| Add interleaved CoT SFT after text-only CoT SFT (ablation) | MINT-CoT training · MMStar-Math | accuracy: 67.6 -> 68.0 | Helped |
| Add interleaved CoT SFT after text-only CoT SFT (ablation) | MINT-CoT training · GeoQA | accuracy: 59.02 -> 62.07 | Helped |
| Add interleaved CoT SFT after text-only CoT SFT (ablation) | MINT-CoT training · MathVista-Math | accuracy: 64.07 -> 67.78 | Helped |
| Add interleaved CoT RL after interleaved CoT SFT (ablation) | MINT-CoT training · MMStar-Math | accuracy: 68.0 -> 69.6 | Helped |
| Add interleaved CoT RL after interleaved CoT SFT (ablation) | MINT-CoT training · GeoQA | accuracy: 62.07 -> 64.72 | Helped |
| Add interleaved CoT RL after interleaved CoT SFT (ablation) | MINT-CoT training · MathVista-Math | accuracy: 67.78 -> 73.70 | Helped |
| Interleave the entire original image at each reasoning step (ablation) | Text-only CoT SFT model · MathVista-Math | accuracy: 64.07 -> 40.37 | Hurt |
| Replace fine-grained token selection with rectangular visual regions (ablation) | Text-only CoT SFT model · MathVista-Math | accuracy: 67.78 -> 65.56 | Hurt |
| Use rectangular visual regions instead of fine-grained token selection (ablation) | Text-only CoT SFT model · MathVista-Math TQA | accuracy: 69.35 -> 70.97 | Helped |
Surprising findings
- Simply interleaving the entire original image at each step (Original Image CoT SFT) substantially hurts MathVista-Math accuracy (40.37 vs 64.07 for text-only SFT).
- Bounding-box interleaving underperforms text-only SFT on GEO and GPS subsets of MathVista-Math.
Limitations
- Relies on GPT-4o for dataset annotation and key-word extraction, so label quality depends on that model.
- Evaluated only on mathematical benchmarks with a single 7B backbone.
- Threshold and projector design chosen by ablation on the test benchmarks; RL stage lacks ground-truth token supervision for evaluation.
Experimental setup
Datasets: Mulberry-260K MINT-CoT dataset Geo170K test set
Benchmarks: MathVista-Math GeoQA MMStar-Math
Metrics: accuracy
Baselines: Qwen2-VL-7B-Instruct Visual-CoT Visual SKETCHPAD ICoT R1-V Open-R1-Multimodal Hint-GRPO MM-Eureka R1-VL-7B Mulberry
Builds on: Chain-of-Thought GRPO Qwen2-VL Mulberry
Closest papers by meaning
- CogFlow: Bridging Perception and Reasoning through Knowledge Internalization for Visual Mathematical Problem Solving (ICLR 2026 · similarity 0.92)
- TVI-CoT: Text-Visual Interleaved Chain-of-Thought Reasoning for Multimodal Understanding (ICML 2026 · similarity 0.92)
- Zebra-CoT: A Dataset for Interleaved Vision-Language Reasoning (ICLR 2026 · similarity 0.92)
- ViRC: Enhancing Visual Interleaved Mathematical CoT with Reason Chunking (CVPR 2026 · similarity 0.91)
- Benchmarking Multimodal Mathematical Reasoning with Explicit Visual Dependency (NeurIPS 2026 · similarity 0.91)
- VisioMath: Benchmarking Figure-based Mathematical Reasoning in LMMs (ICLR 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.