Research map / Architectures and efficiency / RNNs and normalization
Signal in the Noise: Polysemantic Interference Transfers and Predicts Cross-Model Influence
Bofan Gong, Shiyang Lai, James Evans, Dawn Song
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| prepend high-interference tokens from small models instead of random tokens | Pythia-70M · Prompt injection, Locations target | success rate (%): 35.06% -> 36.93% | Improved |
| prepend high-interference tokens instead of random tokens | Llama-3.1-70B-Instruct · Prompt injection, Number target | success rate (%): 7.09% -> 8.85% | Improved |
| prepend high-interference tokens instead of random tokens | Llama-3.1-8B-Instruct · Prompt injection, Locations target | success rate (%): 18.24% -> 20.78% | Improved |
| prepend high-interference tokens instead of random tokens | Gemma-2-9B-Instruct · Prompt injection, Science target | success rate (%): 20.15% -> 20.40% | No clear effect |
| prepend high-interference tokens instead of random tokens | GPT-2-Small · Prompt injection, Number target | success rate (%): 34.71% -> 30.33% | No clear effect |
| prepend high-interference tokens instead of random tokens | Pythia-70M · Prompt injection, Science target | success rate (%): 28.58% -> 23.13% | No clear effect |
| prepend low-interference tokens instead of random tokens | Llama-3.1-70B-Instruct · Prompt injection, Locations target | success rate (%): 24.48% -> 23.09% | No clear effect |
| use token-gradient steering instead of SAE feature direction steering | Pythia-70M and GPT-2-Small · next-token steering | effect size: ~10x larger | Improved |
| use prompt injection instead of direct latent-space steering | Pythia-70M and GPT-2-Small · next-token steering | effect size: ~10% level | Hurt |
| choose high-interference over low-interference matched pair, judged by LLM | GPT-5-mini · high-interference vs low-similarity-matched pairs | pair choice rate: 64.3% | Improved |
| amplify activations of high-polysemanticity super-neurons instead of masking them (ablation) | GPT-2-Small and Pythia-70M · neuron intervention | weighted cosine change: masking less influence; amplifying exponentially greater | Mixed |
Closest papers by meaning
- Capturing Polysemanticity with PRISM: A Multi-Concept Feature Description Framework (NeurIPS 2025 · similarity 0.87)
- Large Language Models as Model Organisms for Human Associative Learning (NeurIPS 2025 · similarity 0.87)
- Rethinking Evaluation of Sparse Autoencoders through the Representation of Polysemous Words (ICLR 2025 · similarity 0.86)
- Talking Heads: Understanding Inter-Layer Communication in Transformer Language Models (NeurIPS 2024 · similarity 0.86)
- Task-Specific Data Selection for Instruction Tuning via Monosemantic Neuronal Activations (NeurIPS 2025 · similarity 0.85)
- Wasserstein Distances, Neuronal Entanglement, and Sparsity (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
