AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

Attend to the Active: Structure-Aware Dynamic Attention in LLMs for Compositional Instruction Following

Fangrui Lv, Yulei Qin, Ruixin Hong, Jian Liang, Jinyang Wu, Ke Li, Xing Sun, Changshui Zhang

ICLR 2026 · Large language models › Language, safety and interpretability · approach: Task vectors and steering

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add structure information for restricting steering spans (ablation)ATA (Llama-3-8B) · Chain/Branch/Parallelaccuracy: Parallel 60.35 -> 69.91; Chain 60.45 -> 62.98; Branch 55.28 -> 58.74Improved
add mutually exclusive masking (ablation)ATA (Llama-3-8B) · Chain/Branch/Parallelaccuracy: Parallel 67.42 -> 69.91; Chain 61.82 -> 62.98; Branch 57.03 -> 58.74Improved
add dynamic attention masking (ablation)ATA (Llama-3-8B) · Chain/Branch/Parallelaccuracy: Parallel 64.31 -> 69.91; Chain 60.74 -> 62.98; Branch 56.65 -> 58.74Improved
add active control for sub-task identification (ablation)ATA (Llama-3-8B) · Chain/Branch/Parallelaccuracy: Parallel 66.82 -> 69.91; Chain 61.14 -> 62.98; Branch 57.23 -> 58.74Improved
use ATA steering instead of SampleAttentionLlama-3-8B · Branchaccuracy: 51.92 -> 58.74Improved
use ATA steering instead of PASTALlama-3-8B · Parallelaccuracy: 60.43 -> 69.91Improved
steer toward misguided (inactive) sub-tasks (ablation)ATA (Llama-3-8B) · Chain/Branch/Parallelaccuracy: Chain 62.98 -> 56.43; Branch 58.74 -> 47.86; Parallel 69.91 -> 54.12Hurt
steer toward random sub-tasks (ablation)ATA (Llama-3-8B) · Chain/Branch/Parallelaccuracy: Chain 62.98 -> 57.38; Branch 58.74 -> 51.04; Parallel 69.91 -> 56.81Hurt
use entropy confidence instead of max probability for active sub-task identification (ablation)ATA (Llama-3-8B) · Chain/Branch/Parallelaccuracy: Chain 61.74 -> 62.98; Branch 57.91 -> 58.74; Parallel 68.62 -> 69.91Improved
use entropy confidence instead of margin confidence (ablation)ATA (Llama-3-8B) · Chain/Branch/Parallelaccuracy: Chain 61.45 -> 62.98; Branch 58.16 -> 58.74; Parallel 69.27 -> 69.91Improved
apply ATA on nested chain & branch instructionsLlama3-8B · Nested Chain & Branchaccuracy: 48.42 -> 51.95Improved
apply ATA on nested chain & parallel instructionsLlama3-13B · Nested Chain & Parallelaccuracy: 56.48 -> 59.30Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy