Research map / Large language models / Instruction and fine-tuning
MindGYM: What Matters in Question Synthesis for Thinking-Centric Fine-Tuning?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MindGYM, a thinking-centric self-synthesis framework that generates single-hop seed and multi-hop QA data with cognitive prompts, and fine-tunes VLMs on 400 synthetic Chinese text samples to gain on math and multimodal reasoning benchmarks.
Problem
Acquiring transferable, structured thinking abilities in large foundation models without rigid templates, crowd-annotated instruction data, or costly RL/iterative prompting.
Approach
Self-synthesize cognitively guided QA data in four stages (background context, single-hop seed questions, multi-hop composition with Bridging/Comparison/Temporal types, structured extraction with thinking traces), then fine-tune in a staged order from guided answering to autonomous solving.
Key results
- On Qwen2.5-VL-7B, MindGYM-Text (400 samples) reaches overall 57.84 vs 56.84 raw and MathVision 28.62 vs 24.67 raw, exceeding the best baseline LIMO (25.90).
- On InternVL-38B, overall rises from 65.16 to 66.23 with 400 samples; baselines like Open-O1 (4k) drop MathVision to 17.43 on Qwen2.5-VL-7B.
- Data-Juicer analysis reports quality mean 0.93 (7B) and 0.98 (32B) with quality std 0.055 and 0.031, versus 0.11 std for MMEvol-DvQA.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune on 400 MindGYM-Text samples instead of raw weights | Qwen2.5-VL-7B · Overall | Overall: 56.84 -> 57.84 | Helped |
| Fine-tune on 400 MindGYM-Text samples instead of raw weights | Qwen2.5-VL-32B · Overall | Overall: 67.58 -> 68.06 | Helped |
| Fine-tune on 400 MindGYM-Text samples instead of raw weights | InternVL-8B · Overall | Overall: 62.46 -> 64.95 | Helped |
| Fine-tune on 400 MindGYM-Text samples instead of raw weights | InternVL-38B · Overall | Overall: 65.16 -> 66.23 | Helped |
| Add structured cognitive thinking to the synthesis process (ablation) | MindGYM-Text on Qwen2.5-VL-7B · Overall | Avg: 56.36 -> 57.84 | Helped |
| Synthesize data in Chinese rather than English (ablation) | MindGYM-Text on Qwen2.5-VL-7B · Overall | Avg: 56.50 -> 57.84 | Helped |
| Keep the prescribed order of fine-tuning phases (ablation) | MindGYM-Text on Qwen2.5-VL-7B · Overall | Avg: 56.51 -> 57.84 | Helped |
| Allow autonomous relationship-category selection instead of balancing categories (ablation) | MindGYM-Text on Qwen2.5-VL-7B · Overall | Avg: 56.51 -> 57.84 | Helped |
| Use structured multi-step synthesis instead of direct CoT synthesis (ablation) | MindGYM-Text on Qwen2.5-VL-7B · Overall | Avg: 55.86 -> 57.84 | Helped |
| Use structured multi-step synthesis instead of direct ToT synthesis (ablation) | MindGYM-Text on Qwen2.5-VL-7B · Overall | Avg: 55.92 -> 57.84 | Helped |
| Keep the full dataset instead of removing flagged samples (ablation) | MindGYM-Text on Qwen2.5-VL-7B · Overall | Overall Avg: 57.49 -> 57.84 | Helped |
| Use text-only synthesis rather than adding Flux-based image generation (ablation) | MindGYM on Qwen2.5-VL-7B · Overall | Avg: 56.80 -> 57.84 | Helped |
Surprising findings
- Open-O1 (4k) substantially degrades Qwen2.5-VL-7B overall (-6.36) and MathVision (to 17.43) despite 10x more data.
- Chinese-language synthesis outperforms English (57.84 vs 56.50), and removing quality filtering via GPT-4o flags slightly lowers performance (57.49).
Limitations
- Multimodal synthesis relies on static image datasets (ScienceQA, OK-VQA) as anchors, limiting diversity and scalability.
- Each experiment was run once with no error bars, so gains of about 1 point may not be statistically established.
- Baselines are not synthesized under the same self-generation setting; comparison is somewhat non-matched and evaluation uses a GPQA subset of 300 samples.
Experimental setup
Datasets: MindGYM self-synthesized text data (400 per model) LIMO (817) Open-O1 (77k subset, 400 and 4k) MMEvol-SciQA (106) MMEvol-DvQA (4k) ScienceQA OK-VQA
Benchmarks: GSM8K MATH GPQA MMStar MathVista-Mini MathVision-Mini
Metrics: accuracy overall average data quality mean and std (Data-Juicer) GPT-4 win rate on depth and breadth
Baselines: LIMO Open-O1 (400) Open-O1 (4k) MMEvol-SciQA MMEvol-DvQA CoT ToT
Builds on: Thinking-Claude LIMO MMEvol Data-Juicer
Closest papers by meaning
- MindLoom: Composing Thought Modes for Frontier-Level Reasoning Data Synthesis (NeurIPS 2026 · similarity 0.91)
- Thinker: Learning to Think Fast and Slow (NeurIPS 2025 · similarity 0.89)
- Don’t Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models (NeurIPS 2025 · similarity 0.87)
- Rethinking Thinking Tokens: LLMs as Improvement Operators (ICML 2026 · similarity 0.87)
- Thinking on the Fly: Test-Time Reasoning Enhancement via Latent Thought Policy Optimization (ICLR 2026 · similarity 0.87)
- Scaling Prompt Synthesis for Large Language Model Reasoning (ICML 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.