Research map / Large language models / Reasoning and chain of thought

Towards Reasoning-Preserving Unlearning in Multimodal Large Language Models

CVPR 2026 · Large language models › Reasoning and chain of thought · approach: Search and self-correction in reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces RMLLMU-Bench, a reasoning-aware benchmark for unlearning in multimodal reasoning models, and R-MUSE, a training-free inference-time activation steering method that forgets answers and reasoning traces while protecting general reasoning.

Problem

Machine unlearning for reasoning multimodal LLMs must remove sensitive information from both final answers and chain-of-thought, without degrading general reasoning; no benchmark jointly measures reasoning leakage and reasoning retention.

Approach

Builds a span-hybrid unlearning subspace from answer and CoT activation differences between refusal-guided and original generations, projects steering orthogonal to a reasoning retain subspace (RRS), gates steering by RRS alignment, and sets steering strength adaptively via a spherical optimal-transport-derived angle (ACS).

Key results

• On LLaVA-1.5-7B (5% forget), R-MUSE reduces forget-set classification accuracy from 51.70% (vanilla) to 20.50% while keeping retain accuracy at 45.90% (vanilla 46.11%).
• On Qwen-2.5-VL-7B (5% forget), R-MUSE reaches 32.50% forget accuracy with retain accuracy 54.00% (vanilla 54.10%), and reasoning leakage drops from 82.40% to 48.20%.
• Ablation on Qwen-2.5-VL-7B: removing RRS drops retain accuracy from 54.10% to 34.00%.

Surprising findings

• R^2MU, designed for large reasoning models, severely degrades reasoning with degenerate repetition such as repeated 'wait' tokens.
• Standard MLLM unlearning baselines such as GA and NPO leave high reasoning leakage despite lowering answer accuracy.

Limitations

• Benchmark is built on the synthetic MLLMU-Bench profiles with LLM-generated reasoning chains.
• RIL implicit-leakage judgment depends on a single LLM judge (Gemini-2.5-Pro).

Experimental setup

Datasets: MLLMU-Bench RMLLMU-Bench

Benchmarks: RMLLMU-Bench

Metrics: classification accuracy ROUGE-L cloze accuracy Reasoning Information Leakage (RIL) Reasoning Capability Retention (RCR)

Baselines: GA GA_Diff KL_Min NPO MMUnlearner MANU R^2MU

Builds on: MLLMU-Bench R^2MU CAA ITI Representation Engineering

Closest papers by meaning

• RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Models (NeurIPS 2024 · similarity 0.91)
• A Closer Look at Machine Unlearning for Large Language Models (ICLR 2025 · similarity 0.90)
• Learning What to Forget: Improving LLM Unlearning via Learned Token-Level Importance (NeurIPS 2026 · similarity 0.90)
• Wisdom is Knowing What not to Say: Hallucination-Free LLMs Unlearning via Attention Shifting (NeurIPS 2025 · similarity 0.90)
• SHRED: Retain-Set-Free Unlearning via Self-Distillation with Logit Demotion (NeurIPS 2026 · similarity 0.90)
• Cross-Modal Unlearning via Influential Neuron Path Editing in Multimodal Large Language Models (AAAI 2026 · similarity 0.89)