Research map / Vision-language and multimodal / Multimodal LLMs
SAUCE: Selective Concept Unlearning in Vision-Language Models with Sparse Autoencoders
Jiahui Geng, Qing Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SAUCE SAE intervention instead of PO (second-best baseline) | LLaVA-v1.5-7B · Concrete concept unlearning | UAg: 74.6% -> 86.3% | Improved |
| use SAUCE instead of PO on abstract concepts | LLaVA-v1.5-7B · Abstract concept unlearning | UAd: 64.6% -> 87.4% | Improved |
| use SAUCE instead of PO for in-domain retention | LLaVA-v1.5-7B · Concrete concept unlearning | IRA: 84.7% -> 90.6% | Improved |
| use SAUCE instead of PO on abstract tasks for cross-domain retention | LLaVA-v1.5-7B · Abstract concept unlearning | CRA: 84.8% -> 83.4% | Hurt |
| use SAUCE instead of PO for model utility | Llama-3.2-11B-Vision-Instruct · Abstract concept unlearning | MME: 93.1% -> 90.7% | Hurt |
| switch the VLM from LLaVA-v1.5-7B to Llama-3.2-11B-Vision-Instruct | SAUCE · Concrete concept unlearning | UAg: 86.3% -> 93.8% | Improved |
| move from concrete to abstract concepts | SAUCE · LLaVA-v1.5-7B | UAg: 86.3% -> 83.1% | Hurt |
| move from concrete to abstract concepts | KL · LLaVA-v1.5-7B | UAg: 70.5% -> 59.4% | Hurt |
| transfer SAE trained on LLaVA-v1.5-7B to LLaVA-v1.5-13B (same vision encoder) | SAUCE · Concrete concept unlearning | UAg: 86.3% -> 84.3% | No clear effect |
| transfer SAE trained on LLaVA-v1.5-7B to LLaVA-v1.6-Vicuna-7B (different vision encoder) | SAUCE · Concrete concept unlearning | MME: 90.1% -> 40.7% | Hurt |
| increase number of concepts unlearned simultaneously | SAUCE · LLaVA-v1.5-7B multi-concept | utility score | Mixed |
| apply Chinese translation and 4-bit quantization attacks | SAUCE · LLaVA-v1.5-7B concrete | UAd: decline of 5% | Hurt |
Closest papers by meaning
- AUVIC: Adversarial Unlearning of Visual Concepts for Multi-modal Large Language Models (AAAI 2026 · similarity 0.89)
- Single Image Unlearning: Efficient Machine Unlearning in Multimodal Large Language Models (NeurIPS 2024 · similarity 0.89)
- Cascaded Sparse Autoencoders LearnMulti-Level Visual Concepts in Multimodal LLMs (NeurIPS 2026 · similarity 0.88)
- Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignment (ICML 2025 · similarity 0.88)
- The Neglected Tails in Vision-Language Models (CVPR 2024 · similarity 0.87)
- SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoders (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
