Research map / Large language models / Language, safety and interpretability
Learning Dynamics of Attention: Human Prior for Interpretable Machine Reasoning
Wonjae Kim, Yoonho Lee
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace discrete MAC attention with continuous ODE-based attention | MAC · CLEVR | Avg. accuracy: 98.9 -> 98.9 | No clear effect |
| Use 4 reasoning steps instead of 12 | MAC with DAFT · CLEVR | Accuracy: 98.9% at step size 12 -> equal performance with step size 4 | No clear effect |
| Replace Dormand-Prince with Euler solver (ablation) | DAFT MAC · CLEVR | Accuracy: $98.6 \pm 0.2$ -> $98.7 \pm 0.2$ | No clear effect |
| Replace Dormand-Prince with Euler solver (ablation) | DAFT MAC · CLEVR | Time (ms): $365.5 \pm 12.5$ (2.37x) -> $167.9 \pm 1.7$ (1.09x) | Improved |
| Replace Dormand-Prince with Runge-Kutta 4th order (ablation) | DAFT MAC · CLEVR | Accuracy: $98.9 \pm 0.2$ -> $98.9 \pm 0.2$ | No clear effect |
| Replace Dormand-Prince with Runge-Kutta 4th order (ablation) | DAFT MAC · CLEVR | Time (ms): $365.5 \pm 12.5$ (2.37x) -> $189.7 \pm 1.9$ (1.23x) | Improved |
| Use a single-step Euler residual connection to restrict attention transitions (ablation) | MAC · CLEVR | accuracy | Hurt |
| Add TLT to the objective with a Lagrange multiplier (ablation) | MAC · CLEVR | performance | Hurt |
| Reduce reasoning steps from four to two (ablation) | DAFT MAC · CLEVR, Query Attribute | accuracy: 99.6 -> 99.3 | No clear effect |
| Reduce reasoning steps from four to two (ablation) | MAC · CLEVR, Query Attribute | accuracy: 99.6 -> 97.5 | Hurt |
Closest papers by meaning
- CoFFT: Chain of Foresight-Focus Thought for Visual Language Models (NeurIPS 2025 · similarity 0.87)
- Stop When Further Reasoning Won’t Help: Attention-State Adaptive Generation in Reasoning Models (ICML 2026 · similarity 0.86)
- From Narrow to Panoramic Vision: Attention-Guided Cold-Start Reshapes Multimodal Reasoning (ICLR 2026 · similarity 0.86)
- Look-Back: Implicit Visual Re-focusing in MLLM Reasoning (AAAI 2026 · similarity 0.86)
- DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning (ICLR 2026 · similarity 0.86)
- Multi-step Visual Reasoning with Visual Tokens Scaling and Verification (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
