Research map / Large language models / Language, safety and interpretability
SLASH the Sink: Sharpening Structural Attention Inside LLMs
Yiming Liu, Bin Lu, Xinbing Wang, Chenghu Zhou, Meng Jin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add SLASH attention sharpening | Qwen3-8B · GraphInstruct (avg) | accuracy: 0.174 -> 0.367 | Improved |
| add SLASH attention sharpening | Llama-3.2-3B · GraphInstruct (avg) | accuracy: 0.085 -> 0.214 | Improved |
| add SLASH attention sharpening | Llama-3.1-8B · GraphInstruct (avg) | accuracy: 0.282 -> 0.352 | Improved |
| add SLASH attention sharpening | Qwen3-4B · GraphInstruct (avg) | accuracy: 0.151 -> 0.195 | Improved |
| add SLASH attention sharpening | GraphWiz fine-tuned models · GraphInstruct (avg) | accuracy: 0.544 -> 0.544 (Mistral-7B) | No clear effect |
| add SLASH attention sharpening | MolecularGPT · MolecularNet | accuracy | No clear effect |
| add SLASH attention sharpening | Qwen3-8B · MolecularNet HIV | accuracy: 0.003 -> 0.670 | Improved |
| add SLASH attention sharpening | Qwen3-14B · MolecularNet BBBP/HIV | accuracy: 0.430 -> 0.430 (BBBP) | No clear effect |
| add SLASH attention sharpening | Llama-3.1-8B · Graph-SST2 | accuracy: 0.638 -> 0.583 | Hurt |
| add SLASH attention sharpening | Qwen3-8B · Graph-SST2 | accuracy: 0.513 -> 0.613 | Improved |
| intervene on whole layers instead of only identified heads (ablation) | SLASH · GraphInstruct Cycle | accuracy: 0.143 -> 0.905 (Qwen3-8B) | Improved |
| add entropy and concentration head filtering instead of entropy only (ablation) | SLASH · GraphInstruct Bipartite | accuracy: 0.288 -> 0.547 (Llama-3.1-8B) | Improved |
Closest papers by meaning
- Attention Mechanisms Perspective: Exploring LLM Processing of Graph-Structured Data (ICML 2025 · similarity 0.94)
- How Do Large Language Models Understand Graph Patterns? A Benchmark for Graph Pattern Comprehension (ICLR 2025 · similarity 0.91)
- From Sequence to Structure: Uncovering Substructure Reasoning in Transformers (NeurIPS 2025 · similarity 0.91)
- Talk like a Graph: Encoding Graphs for Large Language Models (ICLR 2024 · similarity 0.89)
- UniGTE: Unified Graph–Text Encoding for Zero-Shot Generalization across Graph Tasks and Domains (NeurIPS 2025 · similarity 0.89)
- Can Large Language Models Analyze Graphs like Professionals? A Benchmark, Datasets and Models (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
