Research map / Large language models / Reasoning and chain of thought

Reforming the Mechanism: Editing Reasoning Patterns in LLMs with Circuit Reshaping

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Search and self-correction in reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces Reasoning Editing and REdit, which reshapes attribution-based circuits with contrastive meta-learning and dual-level protection before LoRA editing, improving generality and locality on propositional logic (ContextHub) with Qwen2.5-3B.

Problem

Editing a specific LLM reasoning pattern (e.g., a flawed inference rule) so the fix generalizes across instances of the same pattern while not disturbing other reasoning patterns, a generality-locality trade-off.

Approach

Empirically finds a Circuit-Interference Law (edit interference decreases with circuit distance between reasoning patterns) and actively reshapes attribution circuits via an InfoNCE contrastive objective, meta-learning (Reptile-style), and null-space plus prediction-KL protection, before a standard LoRA edit.

Key results

• On ContextHub Level 1 with Qwen2.5-3B, REdit reaches generality 74.1 and locality 94.3, versus LoRA 63.8 / 84.9 and AlphaEdit 67.9 / 87.0.
• On Level 3, REdit achieves generality 55.0 (best) and locality 94.4 (second to ROME's 94.6).
• On Gemma-3-1B-IT, REdit gives the top generality on all three levels (70.8, 69.1, 65.6).

Surprising findings

• Circuit reshaping alone (without editing) yields modest accuracy gains over the raw model, largest on easier levels.
• ROME shows notably lower edit success rates than other methods despite competitive locality.

Limitations

• Evaluated mainly on a single 3B model and a single propositional logic benchmark (ContextHub).
• Math and date experiments are limited in scope (600-problem TemplateGSM subset, one date template variant).
• Improvements over best baseline are modest (about 2% average) and no statistical variance is reported in the main tables.

Experimental setup

Datasets: ContextHub TemplateGSM Date Understanding

Benchmarks: ContextHub TemplateGSM Date Understanding

Metrics: generality locality edit success

Baselines: BIMT LoRA ROME AlphaEdit

Builds on: Edge Attribution Patching InfoNCE Reptile LoRA AlphaEdit ROME

Closest papers by meaning

• A Implies B: Circuit Analysis in LLMs for Propositional Logical Reasoning (NeurIPS 2025 · similarity 0.88)
• SCOUT: Teaching Pre-trained Language Models to Enhance Reasoning via Flow Chain-of-Thought (NeurIPS 2025 · similarity 0.88)
• Mitigating Overthinking in Large Reasoning Language Models via Reasoning Path Deviation Monitoring (NeurIPS 2026 · similarity 0.88)
• Knowledge Circuits in Pretrained Transformers (NeurIPS 2024 · similarity 0.88)
• ReGenesis: LLMs can Grow into Reasoning Generalists via Self-Improvement (ICLR 2025 · similarity 0.88)
• Facts Don't Speak Louder than Words: The Behavioral Essence of Long-CoT Distillation (NeurIPS 2026 · similarity 0.88)