AI Paper Atlas

Research map / Large language models / Reasoning and chain of thought

Correcting in Hindsight: Editing Past Key-Value States for Robust LLM Reasoning

Mengfei Zhang, Yu Mi, Leijing Zhou

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes HEdit, a lightweight paradigm that edits past key-value states during generation so LLMs can correct early reasoning errors.

Problem

Autoregressive LLMs often fail on complex reasoning because early-stage errors cannot be corrected in later steps, due to the irreversibility of the Transformer's cached states.

Approach

Equips models with a hindsight-like capability for dynamic error correction by editing previously cached key-value states during generation, rather than only appending new tokens.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add HEdit KV-cache editingQwen3-0.6B · Math averageaccuracy (average): 17.5 -> 28.2Improved
add HEdit KV-cache editingQwen3-8B · Math averageaccuracy (average): 66.5 -> 72.8Improved
add HEdit KV-cache editingDeepSeek-R1-Distill-Qwen-14B · Math averageaccuracy (average): 70.1 -> 72.3Improved
use HEdit instead of naive backtrackingQwen3-8B · Math averageaccuracy (average): 67.7 -> 72.8Improved
add naive backtracking without editing KV cachebase models · Math averageaccuracy (average): +0.6% to +1.9%No clear effect
add entropy-and-similarity trigger detection (vs random trigger) (ablation)HEdit · Math averageaccuracy (average): 22.2 -> 28.2Improved
add trigger detection (vs random trigger) (ablation)HEdit on Qwen3-8B · Math averageaccuracy (average): 66.8 -> 72.8Improved
add anchor localization (vs random anchor) (ablation)HEdit on Qwen3-0.6B · Math averageaccuracy (average): 18.2 -> 28.2Improved
add anchor localization (vs random anchor) (ablation)HEdit on Qwen3-8B · Math averageaccuracy (average): 62.1 -> 72.8Improved
add trained MLP editor (vs random perturbation) (ablation)HEdit on Qwen3-0.6B · Math averageaccuracy (average): 12.9 -> 28.2Improved
add trained MLP editor (vs random perturbation) (ablation)HEdit on Qwen3-8B · Math averageaccuracy (average): 51.9 -> 72.8Improved
add trained MLP editor (vs random perturbation) (ablation)HEdit on Llama3.1-8B-Instruct · Math averageaccuracy (average): 9.7 -> 21.3Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy