Research map / Vision-language and multimodal / CLIP and zero-shot
Locate then Correct: Debiasing Attention Heads in CLIP
Wei Jie Yeo, Rui Mao, Moloud Abdar, Erik Cambria, Ranjan Satapathy
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add knowledge injection on the identified target states (ablation) | LTC (MA), ViT-B/16 · Waterbirds | worst-group accuracy: 62.5 -> 73.3 | Improved |
| add mean-ablation of spurious states (ablation) | LTC (KI), ViT-B/16 · Waterbirds | worst-group accuracy: 67.3 -> 73.3 | Improved |
| locate states with the contrastive method instead of random states (ablation) | LTC (R), ViT-B/16 · Waterbirds | worst-group accuracy: 36.9 -> 73.3 | Improved |
| randomize the states ablated and injected (ablation) | LTC, ViT-L/14 · Waterbirds | worst-group accuracy: 74.6 -> 15.2 | Hurt |
| apply knowledge injection to specific heads instead of the whole image representation (ablation) | Roboshot (KI), ViT-B/16 · Waterbirds | worst-group accuracy: 45.6 -> 67.3 | Improved |
| mean-ablation of spurious states over zero-shot (ablation) | ZS, ViT-B/16 · Waterbirds | worst-group accuracy: 49.7 -> 62.5 | Improved |
| tune the selected states on a validation set (LTC*) | LTC, ViT-B/16 · Waterbirds | worst-group accuracy: 61.8 -> 73.3 | Improved |
| tune the selected states on a validation set (LTC*) | LTC, ViT-H/14 · Waterbirds | worst-group accuracy: 73.7 -> 77.4 | Improved |
| apply LTC on top of a JTT-trained classifier | JTT, ViT-B/16 · Waterbirds | worst-group accuracy: 72.3 -> 86.4 | Improved |
| apply LTC on top of a JTT-trained classifier | JTT, ViT-H/14 · Waterbirds | worst-group accuracy: 85.7 -> 89.1 | Improved |
| replace orthogonal projection on the whole representation by LTC | Ortho-Cali, ViT-H/14 · Waterbirds | worst-group accuracy: 18.1 -> 73.7 | Improved |
| apply LTC to multi-class counter-animal data | ZS, ViT-B/16 · CounterAnimal hard | accuracy: 54.8 -> 55.2 | Improved |
Closest papers by meaning
- Head Pursuit: Probing Attention Specialization in Multimodal Transformers (NeurIPS 2025 · similarity 0.86)
- Calibrating Multi-modal Representations: A Pursuit of Group Robustness without Annotations (CVPR 2024 · similarity 0.86)
- VITRIX-CLIPIN: Enhancing Fine-Grained Visual Understanding in CLIP via Instruction-Editing Data and Long Captions (NeurIPS 2025 · similarity 0.85)
- Post-pre-training for Modality Alignment in Vision-Language Foundation Models (CVPR 2025 · similarity 0.85)
- Diffusion Feedback Helps CLIP See Better (ICLR 2025 · similarity 0.85)
- Spurious Feature Eraser: Stabilizing Test-Time Adaptation for Vision-Language Foundation Model (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
