Research map / Vision-language and multimodal / Multimodal LLMs
Attention is Case-Sensitive
Maximilian Dillitzer, Tin Stribor Sohn, Jason Corso, Michael Auerbach
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| uppercase the target against lowercase context (TE1) (ablation) | Qwen2.5-7B-Instruct · Text benchmarks | accuracy: +1.85 pp | Improved |
| alternate the case of the target (TA) (ablation) | Qwen2.5-7B-Instruct · Text benchmarks | accuracy: -2.88 pp | Hurt |
| uppercase the target against lowercase context (TE1) (ablation) | LLaMA-3.1-8B-Instruct · MMLU-Pro | accuracy: 29.59 -> 44.39 | Improved |
| alternate the case of the target (TA) (ablation) | LLaMA-3.1-8B-Instruct · Text benchmarks | accuracy: -13.96 pp | Hurt |
| lowercase target with alternating-case context (ADE2) (ablation) | non-reasoning LLMs · MMLU-Pro | accuracy: -6.84 pp | Hurt |
| uppercase the target (TE1) (ablation) | Qwen2.5-7B-Instruct · SQuADv2 | accuracy: +2.13 pp | Improved |
| alternate the case of the target (TA) (ablation) | Qwen2.5-7B-Instruct · SQuADv2 | accuracy: -0.48 pp | Hurt |
| uppercase the target (TE) (ablation) | Qwen2.5-14B-Instruct · MMLU-Pro | accuracy: +9.06 pp | Improved |
| alternate the case of the target (TA2) (ablation) | Qwen2.5-7B-Instruct · ARC-Challenge | attention mass: 0.061 -> 0.104 | Improved |
| apply TA or TE casing to a reasoning model (ablation) | Qwen3-4B-Thinking-2507 · Text benchmarks | attention mass: +0.23 pp | No clear effect |
| apply TA or TE casing to a reasoning model (ablation) | gpt-oss-20B · Text benchmarks | attention mass: +0.07 pp | No clear effect |
| apply non-standard casing to the text prompt of a VLM (ablation) | VLMs (4 models) · RefCOCOg | target-region attention: +1.55 pp | Improved |
Closest papers by meaning
- Head Pursuit: Probing Attention Specialization in Multimodal Transformers (NeurIPS 2025 · similarity 0.87)
- Unveiling Visual Perception in Language Models: An Attention Head Analysis Approach (CVPR 2025 · similarity 0.87)
- When Language Overrules: Revealing Text Dominance in Multimodal Large Language Models (NeurIPS 2026 · similarity 0.86)
- Seeing but Not Believing: Probing the Disconnect Between Visual Attention and Answer Correctness in VLMs (ICLR 2026 · similarity 0.86)
- Don't Take Things Out of Context: Attention Intervention for Enhancing Chain-of-Thought Reasoning in Large Language Models (ICLR 2025 · similarity 0.86)
- Seeing is Understanding: Unlocking Causal Attention into Modality-Mutual Attention for Multimodal LLMs (ICML 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
