Research map / Vision-language and multimodal / Multimodal LLMs

Dual-Latent Memory Routing for Vision-Language Reasoning

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DLMR, which adds dual learnable latent memories (visual and reasoning), a lightweight LoRA injector, and a GRPO-trained discrete router to a frozen MLLM to improve long-horizon vision-language reasoning.

Problem

MLLMs lose track of early visual evidence and intermediate constraints as generations grow longer under a single growing context, causing performance to degrade on long reasoning chains.

Approach

Keep the MLLM frozen and add two disentangled latent memories (visual evidence and reasoning constraints) that are converted into memory tokens by a LoRA injector and selectively injected at delimiter steps by a router that chooses memory type and token budget.

Key results

• On Qwen2.5-VL-7B, DLMR (RL) reaches 56.45 average on four reasoning benchmarks versus 50.76 for Visual-RFT, and DLMR (SFT) reaches 71.45 general average versus 68.38 for Visual-RFT.
• On InternVL-3-8B, DLMR (SFT) reaches 79.25 general average and 59.04 reasoning average, versus 76.04 and 55.02 for Visual-RFT.
• Dual latent memory ablation on Qwen2.5-VL-7B raises six-benchmark average from 52.05 to 59.73 over a single shared memory; MathVision improves from 22.37 to 35.32.

Surprising findings

• The router uses the reasoning memory more often on reasoning tasks and the visual memory more often on general tasks, with injection frequency tapering near the end of decoding.
• DLMR trained on multimodal data transfers to text-only benchmarks, e.g. GSM8K rising from 68.08 to 83.47.

Limitations

• Relies on a fixed set of delimiter tokens and a per-sample injection cap for routing eligibility, which is a heuristic design choice.
• Training data includes the evaluated benchmarks' training splits, so gains may partly reflect in-domain training.
• Compute hardware and training cost are not reported.

Experimental setup

Datasets: MMVet MMStar RealWorldQA MMMU MathVerse MathVision MathVista OpenMMReasoner

Benchmarks: MMVet MMStar RealWorldQA MMMU MathVerse MathVision MathVista MMLU GSM8K ARC-Challenge LogicVista OCR-Bench

Metrics: accuracy average generated tokens inference latency

Baselines: Vanilla CoT CCoT SFT GRPO Visual-RFT RCTS-RAG

Builds on: GRPO Visual-RFT LoRA chain-of-thought

Closest papers by meaning

• Vision-aligned Latent Reasoning for Multi-modal Large Language Model (ICML 2026 · similarity 0.92)
• VisMem: Latent Vision Memory Unlocks Potential of Vision-Language Models (CVPR 2026 · similarity 0.91)
• SLVR: Structured Latent Visual Reasoning via Human-like Reasoning Flows (NeurIPS 2026 · similarity 0.91)
• Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models (CVPR 2025 · similarity 0.91)
• Towards General Continuous Memory for Vision-Language Models (NeurIPS 2025 · similarity 0.91)
• POINTS-Long: Adaptive Dual-Mode Visual Reasoning in MLLMs (CVPR 2026 · similarity 0.91)