Research map / Large language models / Instruction and fine-tuning
Can MLLMs Absorb Math Reasoning Abilities from LLMs as Free Lunch?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes IP-Merging, a tuning-free method that merges math-reasoning parameters from off-the-shelf math LLMs into MLLMs by selecting layers via SVD subspace similarity, rescaling, and projecting them into the MLLM subspace.
Problem
MLLMs lag behind LLMs in math reasoning, and collecting multimodal math data or tuning is costly; naive model merging of math LLMs into MLLMs fails due to parameter-space misalignment between the two.
Approach
Identify reasoning-associated layers by SVD-based corresponding-angle similarity between task vectors of the MLLM and math LLM, then rescale the LLM task vector by a nuclear-norm ratio and project it onto the MLLM's singular subspace before adding it to the MLLM weights.
Key results
- On MathVerse, IP-Merging raises LLaVA-1.5-7B average accuracy from 11.3 to 15.3 (+4.0), outperforming Task Arithmetic, TIES and EMR merging.
- With Qwen2-VL-7B-Instruct as base, IP-Merging improves MathVerse overall from 24.8 to 28.5 and MathVista from 55.4 to 60.2.
- IP-Merging keeps general benchmarks stable (e.g., Qwen2-VL MMMU 50.7 to 50.7, TextVQA 83.8 to 84.0), whereas Task Arithmetic and TIES degrade them substantially.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| merge math LLM parameters with IP-Merging | LLaVA-1.5-7B · MathVerse Overall | accuracy: 11.3 -> 15.3 | Helped |
| merge math LLM parameters with IP-Merging | Qwen-2-VL-7B-Instruct · MathVista Overall | accuracy: 55.4 -> 60.2 | Helped |
| merge math LLM parameters with IP-Merging | InternVL3-8B-Instruct · MathVista Overall | accuracy: 66.1 -> 67.6 | Helped |
| merge math LLM parameters with IP-Merging | LLaVA-1.5-7B · MMMU | accuracy: 34.2 -> 34.4 | No clear effect |
| add parameter selection (ablation) | IP-Merging on LLaVA-1.5-7B · MathVista | accuracy: 25.2 -> 25.8 | Helped |
| add rescaling (ablation) | IP-Merging on LLaVA-1.5-7B · MathVista | accuracy: 25.2 -> 26.2 | Helped |
| add projection (ablation) | IP-Merging on LLaVA-1.5-7B · MathVista | accuracy: 25.2 -> 25.6 | Helped |
| combine parameter selection and projection (ablation) | IP-Merging on LLaVA-1.5-7B · MathVista | accuracy: 26.3 | Helped |
| combine parameter selection and rescaling (ablation) | IP-Merging on LLaVA-1.5-7B · MathVista | accuracy: 26.3 | Helped |
| combine rescaling and projection (ablation) | IP-Merging on LLaVA-1.5-7B · MathVista | accuracy: 26.7 | Helped |
| use parameter selection, rescaling, and projection together (ablation) | IP-Merging on LLaVA-1.5-7B · MathVista | accuracy: 28.2 | Helped |
| vary the similarity threshold from 0.1 to 0.9 (ablation) | IP-Merging on LLaVA-1.5-7B · MathVista | accuracy: 25.4; 26.2; 26.4; 28.4; 26.4; 27.3; 27.6; 27.0; 27.3 | Mixed |
Surprising findings
- Naive task arithmetic merging of a math LLM into an MLLM degrades math performance (LLaVA MathVerse 11.3 to 9.8).
- TIES merging collapses TextVQA for LLaVA (47.5 to 1.1) and Qwen2-VL (83.8 to 3.1).
Limitations
- Only effective when the MLLM and math LLM have the same size and share the same foundation LLM architecture.
- Merging can degrade other capabilities for baselines; IP-Merging gains on GPS for LLaVA are absent (21.6 vs 22.1 base).
Experimental setup
Benchmarks: MathVista MathVerse DynaMath MathVision MMMU TextVQA MMBench
Metrics: accuracy
Baselines: Task Arithmetic Ties Merging EMR Merging
Builds on: task arithmetic TIES-Merging EMR-Merging DARE
Closest papers by meaning
- MathFimer: Enhancing Mathematical Reasoning by Expanding Reasoning Steps through Fill-in-the-Middle Task (ICLR 2026 · similarity 0.90)
- MathSE: Improving Multimodal Mathematical Reasoning via Self-Evolving Iterative Reflection and Reward-Guided Fine-Tuning (AAAI 2026 · similarity 0.89)
- GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models (ICLR 2025 · similarity 0.89)
- MV-MATH: Evaluating Multimodal Math Reasoning in Multi-Visual Contexts (CVPR 2025 · similarity 0.89)
- Putnam-AXIOM: A Functional & Static Benchmark for Measuring Higher Level Mathematical Reasoning in LLMs (ICML 2025 · similarity 0.89)
- AbstRaL: Augmenting LLMs' Reasoning by Reinforcing Abstract Thinking (ICLR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.