Research map / Large language models / Language, safety and interpretability

Decomposing how prompting steers behavior

NeurIPS 2026 · Large language models › Language, safety and interpretability · approach: Reasoning mechanisms and failure modes · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a nested geometric decomposition (translation, rigid+uniform scale, rigid+axis scale, affine, nonlinear) of prompt-induced hidden-state change, fit by cross-validated Procrustes/ridge/MLP maps and tested by single-layer causal patching across 3 LLMs, 3 VLMs and 6 datasets.

Problem

How a change in instruction prompt reshapes internal representations to steer behavior, and what class of transformation best explains prompt-induced representational change.

Approach

Treats prompting as a stimulus-invariant map between hidden-state manifolds under two prompts, fitting a strictly nested hierarchy of transformation classes and attributing cross-validated variance to each tier, then testing each fitted map via causal single-layer patching for both geometry and behavior.

Key results

• Translation accounts for the largest share of cross-validated explained variance, with higher tiers contributing smaller increments mainly in mid-to-deep layers.
• The incremental contribution hierarchy is reported as translation > affine > rigid+uniform scaling > rigid+axis scaling ≈ nonlinear, consistent across prompt-pair groups.
• In causal patching, affine maps nearly recover target-prompt task geometry and produce larger behavioral gains than translation or rigid tiers; nonlinear gives the strongest recovery but only slightly above affine.

Surprising findings

• Translation and uniform-scaled rigid maps leave distance-based RDM rankings and silhouette scores unchanged under causal intervention.
• Nonlinear maps do not substantially outperform affine maps in explaining geometry or behavior.

Limitations

• Analysis limited to zero-shot instruction prompts; no few-shot, soft prompts, or long-context conditioning.
• Causal intervention replaces a single layer at the final input token only; multi-layer or multi-token interventions not tested.
• Behavioral evaluation relies on keyword-based evaluators, and the nonlinear tier is a single shared MLP so its gains are a lower bound.

Experimental setup

Datasets: EmotionalStory WritingStyle Number EmoSet StyleTransfer COCO

Benchmarks: EmotionalStory WritingStyle Number EmoSet StyleTransfer COCO

Metrics: cross-validated incremental R^2 RDM Spearman correlation silhouette score behavioral relevance behavioral accuracy

Baselines: prompt B patching (oracle) translation tier axis-scaled rigid tier

Builds on: Procrustes analysis representational similarity analysis activation steering

Closest papers by meaning

• Do different prompting methods yield a common task representation in language models? (NeurIPS 2025 · similarity 0.88)
• Finding Interpretable Prompt-Specific Circuits in Language Models (NeurIPS 2026 · similarity 0.86)
• Enhancing Visual Prompting through Expanded Transformation Space and Overfitting Mitigation (NeurIPS 2025 · similarity 0.86)
• When Do Prompting and Prefix-Tuning Work? A Theory of Capabilities and Limitations (ICLR 2024 · similarity 0.86)
• Understanding Prompt Tuning and In-Context Learning via Meta-Learning (NeurIPS 2025 · similarity 0.85)
• Revisiting the Power of Prompt for Visual Tuning (ICML 2024 · similarity 0.85)