Research map / Label-efficient and robust learning / Knowledge distillation

Subliminal Learning Is Steering Vector Distillation

NeurIPS 2026 · Label-efficient and robust learning › Knowledge distillation · approach: Teacher properties and robustness · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that subliminal learning in LLMs is mediated by a single steering vector: a system-prompted teacher's effect is approximated by a residual-stream steering vector that the LoRA student learns, generalized as steering vector distillation with adaptive optimizers being necessary.

Problem

Subliminal learning—where a student fine-tuned on semantically unrelated teacher outputs acquires the teacher's semantic trait—remains mechanistically unexplained, including why only some traits transfer and why transfer requires the same base model.

Approach

Treat the system-prompted teacher as a model with a steering vector added to its activations; the student's fine-tuning installs an aligned residual-stream direction. Generalize to steering vector distillation (random and semantic vectors) and show Adam's per-parameter scaling is needed because SGD is dominated by outlier LoRA gradients.

Key results

• Fine-tuning installs a residual-stream direction in the student whose empirical activation similarity with the teacher vector increases over training, while clean-data controls stay near 0.1.
• Inference-time steering rate predicts trained-student animal affinity; traits that do not steer the reference model (racoons, rabbits, giraffes, frogs) are not subliminally learned.
• Replacing the student's v_student component with the reference model's removes over 50% of trait-aligned behavior; SGD fails to install the steering direction while Adam succeeds.

Surprising findings

• Full fine-tuning fails to induce trait affinity beyond the reference model's existing preferences, so subliminal learning appears low-rank specific.
• Gemma-3-4b-it shows significantly weaker subliminal learning than Qwen2.5-7B-Instruct.

Limitations

• Steering vectors do not fully account for effect sizes; adding v_student does not fully recover Qwen's cat affinity and ablation does not eliminate it.
• Only system-prompt teachers studied with limited data points; per-coordinate adaptive scaling necessity lacks a principled explanation.
• Results on extended 10-epoch Qwen training are less salient.

Experimental setup

Datasets: number sequences generated by steered/system-prompted teacher code templates Alpaca (paraphrased) 50 evaluation preference questions per trait

Benchmarks: animal and persona preference evaluation questions semantic shift rate on natural prompts

Metrics: trait expression rate empirical activation similarity (cosine) activation-difference cosine similarity relative cross-entropy loss reduction semantic shift rate AUC

Baselines: token entanglement explanation divergence tokens explanation full fine-tuning SGD SGD with momentum RMSProp

Builds on: subliminal learning difference-of-means steering LoRA

Closest papers by meaning

• Improving Instruction-Following in Language Models through Activation Steering (ICLR 2025 · similarity 0.87)
• Subliminal Transfer of Unsafe Behaviors in AI Agent Distillation (NeurIPS 2026 · similarity 0.86)
• Analysing the Generalisation and Reliability of Steering Vectors (NeurIPS 2024 · similarity 0.86)
• When Is Rank-1 Steering Cheap? Geometry, Granularity, and Budgeted Search (NeurIPS 2026 · similarity 0.85)
• Progressive distillation induces an implicit curriculum (ICLR 2025 · similarity 0.85)
• The Sharp Directions Are Against You: Curvature Analysis of Activation Steering (NeurIPS 2026 · similarity 0.85)