Research map / Large language models / Reasoning and chain of thought

The Master Key Hypothesis: Unlocking Cross-Model Capability Transfer via Linear Subspace Alignment

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: Theory of autoregressive reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes UNLOCK, a training-free, label-free method that extracts a capability direction from activation differences between Source variants (e.g., CoT vs direct prompting, or base vs instruct) and transfers it to another model via low-rank linear alignment for inference-time steering, improving reasoning across Qwen, OLMo-2, gemma-2, Ministral-3 models.

Problem

Capabilities such as chain-of-thought and math reasoning are costly to learn in each new model and hard to reuse across models of different sizes; existing cross-model transfer methods need weight compatibility, multi-model logit computation, or labeled contrastive data.

Approach

Extract a mean or PCA steering direction from unlabeled activation differences between a capability-present and capability-absent Source pair, align the Source and Target hidden spaces via a rank-k least-squares linear map over SVD subspaces, and apply the lifted direction as a norm-preserving intervention on the Target's hidden states at inference time.

Key results

• Transferring CoT direction from Qwen1.5-14B to Qwen1.5-7B raises MATH accuracy from 8.0% to 20.1% under direct prompting, above 18.2% of the 7B chat model with CoT prompting.
• Transferring a math reasoning direction from Qwen3-4B (instruct vs base) to Qwen3-14B-Base improves AGIEval Math from 61.1% to 71.3%, surpassing the 67.8% of the 14B instruct model.
• Gains are asymmetric: small-to-large transfer yields larger improvements; for gemma-2 small-to-large CoT transfer improves the 9B model by 44.4% on average while large-to-small gives only 1.6%.

Surprising findings

• Large-to-small CoT transfer into gemma-2-2B yields almost no gain (GSM8K 9.5 vs 5.8 baseline), suggesting capability cannot be introduced when absent in the target.
• Task-agnostic vs task-conditioned preference reverses by transfer direction: task-conditioned wins for large-to-small, task-agnostic for small-to-large.

Limitations

• Gains depend strongly on hyperparameters (aggregator, rank k, alpha, n) chosen by grid search, with no universal optimum across models.
• Transfer is much weaker when the capability is absent in the target model, e.g., gemma-2 and OLMo-2 CoT transfer falls short of post-trained models.
• The Master Key Hypothesis is an empirical operational hypothesis without a mechanistic account; the authors note the atomicity definitions are incomplete.

Experimental setup

Datasets: GSM8K MATH SVAMP AGIEval-Math DeepMind-Math Minerva-Math OlympiadBench

Benchmarks: GSM8K MATH SVAMP AGIEval-Math DeepMind-Math Minerva-Math OlympiadBench

Metrics: accuracy generation length reconstruction error spectral entropy

Baselines: Direct prompting of Target base model CoT prompting of Target base model post-trained Target model Proxy tuning Task vectors

Builds on: Linear Representation Hypothesis Platonic Representation Hypothesis task vectors proxy tuning

Closest papers by meaning

• RAST: Reasoning Activation in LLMs via Small-model Transfer (NeurIPS 2025 · similarity 0.85)
• Tracing the Traces: Latent Temporal Signals for Efficient and Accurate Reasoning (ICLR 2026 · similarity 0.84)
• When Models Know More Than They Can Explain: Quantifying Knowledge Transfer in Human-AI Collaboration (NeurIPS 2025 · similarity 0.84)
• Task Vector Geometry Underlies Dual Modes of Task Inference in Transformers (NeurIPS 2026 · similarity 0.84)
• Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? (NeurIPS 2025 · similarity 0.84)
• Prescriptive Scaling Reveals the Evolution of Language Model Capabilities (ICML 2026 · similarity 0.84)