Research map / Label-efficient and robust learning / Recognition and long-tail
Learning to Intervene on Concept Bottlenecks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Concept Bottleneck Memory Models (CB2Ms), which add a two-fold memory of past interventions and mistakes to concept bottleneck models so that interventions can be reapplied to similar unseen inputs and mistakes can be detected before human feedback.
Problem
Concept bottleneck model interventions are typically applied once to a single sample and then discarded, requiring repeated human interaction; there is also no way to target which samples need intervention.
Approach
Augment a CBM with a mistake memory (encodings of past bottleneck-error samples, compared by Euclidean distance) coupled with an intervention memory, so that a stored human intervention is reapplied to new inputs whose encodings are close to a remembered mistake, and the same memory flags likely mistakes for targeted querying.
Key results
- On CUB (Aug.), CB2M improves identified-instance concept accuracy from 86.4% (CBM) to 99.0% and class accuracy from 5.0% to 88.7%.
- On Parity C-MNIST mistake detection, CB2M reaches AUROC 83.4 vs 65.7 for softmax.
- Under MNIST-to-SVHN distribution shift, CB2M raises full-set class accuracy from 51.0% to 57.3%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| reuse human interventions from the memory on similar examples | CBM via CB2M · CUB (Aug.), Id | class accuracy: 5.0\pm1.7 -> \mathbf{88.7}\pm 5.4 | Helped |
| reuse human interventions from the memory on similar examples | CBM via CB2M · Parity MNIST (unbalanced), Id | class accuracy: 22.5\pm5.7 -> \mathbf{93.7}\pm1.9 | Helped |
| reuse human interventions from the memory on similar examples | CBM via CB2M · Parity C-MNIST, Id | class accuracy: 20.1\pm7.1 -> \mathbf{85.9}\pm4.7 | Helped |
| use CB2M rather than softmax confidence for mistake detection | CBM mistake detection · CUB (conf.) | AUROC: 77.4 \pm 1.1 -> \mathbf{85.1} \pm 0.5 | Helped |
| use CB2M rather than softmax confidence for mistake detection | CBM mistake detection · Parity C-MNIST | AUROC: 65.7\pm0.3 -> \mathbf{83.4}\pm0.8 | Helped |
| use CB2M rather than softmax confidence for mistake detection | CBM mistake detection · Parity MNIST (unbalanced) | AUROC: \mathbf{90.7}\pm1.7 -> 88.7\pm0.4 | Hurt |
| select interventions using CB2M mistake detection rather than softmax | CBM · CUB, Full Set | NRI: 70.1 \pm 3.1 -> \mathbf{75.5} \pm 4.5 | Helped |
| select interventions using CB2M mistake detection rather than softmax | CBM · Parity MNIST (unbalanced), Full Set | NRI: 62.1\pm4.9 -> \mathbf{69.6}\pm4.1 | Helped |
| select interventions using CB2M mistake detection rather than softmax | CBM · Parity C-MNIST, Full Set | NRI: 87.3\pm0.8 -> \mathbf{89.7}\pm6.1 | Helped |
| generalize interventions to unseen SVHN examples after training on Parity MNIST | CBM via CB2M · Parity MNIST to SVHN, Full set | class accuracy: $51.0\pm0.1$ -> $57.3\pm0.2$ | Helped |
| increase memory from 25% to larger fractions of available intervention data (ablation) | CB2M · CUB (Aug.) | concept and class accuracy | Helped |
| increase memory size (ablation) | CB2M · identified instances across datasets | accuracy | Hurt |
Surprising findings
- Fine-tuning the CBM on intervention data is competitive with CB2M on full-set accuracy in several settings, notably Parity C-MNIST (95.0 vs 88.4 concept accuracy).
- On the near-perfect unbalanced/original MNIST base model, CB2M's mistake detection is worse than softmax (AUROC 64.6 vs 93.3).
Limitations
- Assumes human interventions are correct; malicious feedback could poison the memory.
- Memory search and storage may limit scalability in large-scale settings.
- High false negative rates for reapplied interventions (e.g., FNR ~87% on CUB).
Experimental setup
Datasets: CUB MNIST ColorMNIST (C-MNIST) SVHN
Benchmarks: CUB CUB (Aug.) CUB (conf.) Parity MNIST (unbalanced) Parity C-MNIST Parity MNIST to SVHN
Metrics: concept accuracy class accuracy AUROC AUPR normalized relative improvement (NRI) FPR FNR
Baselines: CBM CBM (ft) Random Softmax ECTP
Builds on: concept bottleneck models concept intervention (Koh et al.) neural nearest neighbors
Closest papers by meaning
- Stochastic Concept Bottleneck Models (NeurIPS 2024 · similarity 0.91)
- Energy-Based Concept Bottleneck Models: Unifying Prediction, Concept Intervention, and Probabilistic Interpretations (ICLR 2024 · similarity 0.90)
- Causally Reliable Concept Bottleneck Models (NeurIPS 2025 · similarity 0.90)
- Concepts Worth Having: Refining VLM-Guided Concept Bottleneck Models with Minimal Annotations (NeurIPS 2026 · similarity 0.89)
- Counterfactual Concept Bottleneck Models (ICLR 2025 · similarity 0.89)
- VLG-CBM: Training Concept Bottleneck Models with Vision-Language Guidance (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.