Research map / Large language models / Reasoning and chain of thought
L2V-CoT: Cross-Modal Transfer of Chain-of-Thought Reasoning via Latent Intervention
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes L2V-CoT, a training-free method that extracts low-frequency CoT direction representations from DeepSeek-R1-Distill-Qwen-32B, resamples them to match dimensions, and injects them into VLM hidden states at inference to transfer chain-of-thought reasoning.
Problem
VLMs struggle with multi-step reasoning due to scarce multimodal reasoning data, and existing LLM-to-VLM reasoning transfer methods either need costly training or require architecturally aligned models.
Approach
Linear Artificial Tomography shows LLM and VLM share similar low-frequency CoT direction representations; low-pass filtering plus frequency-domain resampling (LMN) of the LLM's CoT direction vector allows dimension matching, and the result is injected into a middle layer of the VLM at inference with norm-preserving rescaling.
Key results
- On Qwen2-VL-7B-Instruct, L2V-CoT reaches 64.2 on MathVista vs. 60.5 for non-CoT response and 63.7 for Finetuned CoT.
- Average improvement across benchmarks is 3.7% with gains up to 8.6%, per the conclusion.
- Ablations show interpolation instead of LMN resampling degrades performance (e.g., QwenVL MathVista 49.5), and injecting VLM's own low-pass representation gives smaller gains than LLM representation (61.4 vs 64.2).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| inject low-pass LLM CoT direction representation (vs. non-CoT baseline) | LLaVA-Next-LLaMA3-8B, InternVL2-8B, Qwen2-VL-7B-Instruct · MathVista, MathVerse, MMStar, DynaMath, MathVision | accuracy: e.g. MathVista All 35.2 -> 41.8 (LLaVA) | Helped |
| replace LMN frequency resampling with interpolation (ablation) | L2V-CoT · MathVista All, Qwen2-VL-7B-Instruct | accuracy: 64.2 -> 49.5 (below non-CoT 60.5) | Hurt |
| inject VLM's own low-pass direction instead of LLM direction (ablation) | L2V-CoT · MathVista All, Qwen2-VL-7B-Instruct | accuracy: 64.2 -> 61.4 | Hurt |
| use smaller LLM source (DeepSeek-R1-Distill-Qwen-7B vs 32B) (ablation) | L2V-CoT · MathVista All, LLaVA-Next-LLaMA3-8B | accuracy: 41.8 -> 38.6 | Hurt |
| increase number of CoT examples used to extract representation (ablation) | L2V-CoT · MathVerse, LLaVA-Next-LLaMA3-8B | accuracy: performance improves with more examples (qualitative) | Helped |
| Few-shot CoT demonstrations added to prompt | LLaVA, InternVL, QwenVL · MathVista All | accuracy: e.g. Qwen2-VL 60.5 -> 58.4 | Hurt |
| Modal merging with same-backbone LLM | LLaVA, InternVL, QwenVL, Idefics · MathVista All | accuracy: moderate gains, e.g. Qwen2-VL 60.5 -> 61.5 | Helped |
| RoT representation injection from CoT prompts | LLaVA, InternVL, QwenVL, Idefics · MathVista All | accuracy: e.g. Qwen2-VL 60.5 -> 62.9 | Helped |
| L2V-CoT combined with explicit method Mulberry | Mulberry (Llama-11B, Qwen2VL-2B) · Mulberry benchmarks | accuracy: further improvements (figure only) | Helped |
Surprising findings
- Few-shot CoT prompting performs worse than the non-CoT baseline across all VLMs.
- The training-free injection outperforms supervised fine-tuning on the same CoT data across the evaluated VLMs.
Limitations
- Injection strength and layer are hyperparameters that must be tuned per model and per task (MathVerse uses different settings per subset).
- Gains are bounded by the VLM's own capacity; the method relies on a stronger LLM teacher being available.
Experimental setup
Datasets: STILL-2 MathVista MathVerse MathVision DynaMath MMStar
Benchmarks: MathVista MathVerse MMStar DynaMath MathVision
Metrics: accuracy
Baselines: Non-CoT response Few-Shot CoT MathNeuro Modal Merging RoT Finetuned CoT
Builds on: Linear Artificial Tomography RoT (reasoning on tap / representation of thought) Model merging for reasoning transfer Contrast-Consistent Search GLoRE
Closest papers by meaning
- SIM-CoT: Supervised Implicit Chain-of-Thought (ICLR 2026 · similarity 0.91)
- Uni-CoT: Towards Unified Chain-of-Thought Reasoning Across Text and Vision (ICLR 2026 · similarity 0.90)
- MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiency (ICML 2025 · similarity 0.90)
- CoT Vectors: Transferring and Probing the Reasoning Mechanisms of LLMs (ICLR 2026 · similarity 0.90)
- Multi-Modal Latent Space Learning for Chain-of-Thought Reasoning in Language Models (AAAI 2024 · similarity 0.90)
- Rationale-Enhanced Decoding for Multi-modal Chain-of-Thought (CVPR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.