Research map / Large language models / Language, safety and interpretability

Metaphor Is Not All Attention Needs

NeurIPS 2026 · Large language models › Language, safety and interpretability · approach: Safety alignment of LLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Interprets why poetic jailbreaks work by ablating poetic devices and probing a fixed-length, interpretable max-pooled attention representation of Qwen3-14B, finding format is linearly decodable while jailbreak success is weakly decodable.

Problem

Why literary (poetic) reformulations bypass LLM safety alignment: whether success depends on specific poetic devices, on failure to recognize poetic format, or on deeper changes in how models process stylistically irregular prompts.

Approach

Builds a 72-dimensional interpretable attention feature vector (max-pooled over heads, aggregated over 3 generation phases, 4 layer clusters, and 6 functional token groups with mean pooling), then uses clustering and linear probes to test whether attention encodes literary format versus safety outcome; combined with a device-level ablation framework.

Key results

• Format probe (poetry vs prose) reaches 0.985 accuracy and 0.997 AUC with logistic regression on the full dataset.
• Safety probes within each format reach only about 0.66 accuracy (poetry 0.658, prose 0.673 for logistic regression), well above chance but far below the format probe.
• Attack success rate is 46.92% for poetry versus 10.44% for prose on the main dataset.

Surprising findings

• Safety signal in attention is weak and largely separate from the format signal, suggesting safety-relevant computation may lie outside attention (e.g., MLPs or residual stream).
• In ablations, introducing rhyme changed safety labels mainly because it forced replacement of blacklisted vocabulary, not because of rhyme itself.

Limitations

• Analysis limited to one model (Qwen3-14B) with thinking mode disabled and fixed 50-token generation.
• Attention-only analysis; does not examine MLP or residual stream components.
• Max-pooling over heads and mean-pooled aggregation smooth away head-level and peak information; functional taxonomy sensitivity unexplored.

Experimental setup

Datasets: MLCommons AILuminate Benchmark poetic reformulations generated by DeepSeek-R1 (1200 prompts) calibration set of 20 poetry-prose pairs

Benchmarks: MLCommons AILuminate Benchmark (poetry and prose versions)

Metrics: accuracy AUC attack success rate PCA variance explained

Baselines: prose prompts

Builds on: adversarial poetry jailbreak Ward hierarchical clustering

Closest papers by meaning

• Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting (ICLR 2024 · similarity 0.86)
• Unveiling and Manipulating Prompt Influence in Large Language Models (ICLR 2024 · similarity 0.85)
• Distraction is All You Need for Multimodal Large Language Model Jailbreaking (CVPR 2025 · similarity 0.84)
• Cannot See the Forest for the Trees: Invoking Heuristics and Biases to Elicit Irrational Choices of LLMs (ICML 2025 · similarity 0.84)
• Is Your Multimodal Language Model Oversensitive to Safe Queries? (ICLR 2025 · similarity 0.84)
• Within-Model vs Between-Prompt Variability in Large Language Models for Creative Tasks (NeurIPS 2026 · similarity 0.84)