Research map / Large language models / Reasoning and chain of thought

Memory Inception: Latent-Space KV Cache Manipulation for Steering LLMs

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: KV cache compression · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes memory inception, a training-free method that steers LLMs by appending text-derived latent KV banks only at selected attention layers/heads, evaluated on personality steering, dialogue shift, and HARDMath/PHYSICS against prompting and CAA.

Problem

Persistent guidance for steering LLMs is usually either cached as visible prompt tokens at every layer (costly, clutters context) or applied via activation steering (compact but weaker and less fine-grained); the paper asks how to deliver large structured reminders more efficiently.

Approach

Encode reminder text into latent KV memory-bank slots and let only automatically selected attention layers/heads/KV groups attend over them alongside the prompt, using canonical pre-RoPE keys and size-normalized bank-level routing, so guidance is hidden from the transcript and stored at only a few layers.

Key results

• On Qwen3 Big Five self-evaluation, MI achieves DAS 0.270 vs 0.115 for CAA (prompt 0.355).
• On PHYSICS with Qwen3-30B-A3B, MI reaches 76.9% overall vs 75.4% prompt and 73.9% plain, winning 10 of 12 subject-mode cells.
• On Qwen3 dialogue shift, MI raises post-shift alignment to 0.816 vs 0.526 for CAA and 0.438 for prompt-init, with KV storage reduced by up to 118x.

Surprising findings

• On HARDMath, visible prompting (67.3%) outperforms MI (63.8%) overall, though MI beats plain (60.8%).
• On Llama dialogue shift, CAA leads raw alignment (1.554 vs MI 1.516), and on Llama self-evaluation generation prompting beats MI on GAS.

Limitations

• Judging and scoring protocols differ across tasks, so cross-task comparisons are indirect.
• Bank quality and selector calibration are task-dependent and require calibration examples.
• Results are backbone-dependent; KV analysis measures storage footprint, not realized latency or VRAM.

Experimental setup

Datasets: IPIP-50 Big Five items HARDMath PHYSICS GSM8K MMLU

Benchmarks: IPIP-50 Big Five self-evaluation Big Five generation Dialogue shift HARDMath PHYSICS GSM8K MMLU

Metrics: DAS (drift-adjusted score) GAS (generation-adjusted score) post-shift alignment normalized benchmark score win count KV storage ratio

Baselines: Prompting CAA Plain model Prompt-init

Builds on: Contrastive Activation Addition Rotary position embeddings vLLM Spotlight KV Cache Steering

Closest papers by meaning

• Don't Lose Focus: Activation Steering via Key-Orthogonal Projections (NeurIPS 2026 · similarity 0.88)
• When Is Rank-1 Steering Cheap? Geometry, Granularity, and Budgeted Search (NeurIPS 2026 · similarity 0.88)
• Steer2Edit: From Activation Steering to Component-Level Editing (NeurIPS 2026 · similarity 0.88)
• Mitigating Memorization Where It Happens (NeurIPS 2026 · similarity 0.88)
• Focus On This, Not That! Steering LLMs with Adaptive Feature Specification (ICML 2025 · similarity 0.87)
• Improving Instruction-Following in Language Models through Activation Steering (ICLR 2025 · similarity 0.87)