Research map / Label-efficient and robust learning / Recognition and long-tail
Beyond Concept Bottleneck Models: How to Make Black Boxes Intervenable?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a post hoc method to intervene on pretrained black-box networks via probes and latent editing, plus a fine-tuning objective that maximizes intervenability using a small concept-labelled validation set.
Problem
Concept-based instance-specific interventions are only available in concept bottleneck models, which must be trained with concept labels and are often not feasible for pretrained or large black-box networks; black boxes typically do not respond to concept edits.
Approach
Train a probe on a small concept-annotated validation set to map intermediate activations to concepts, edit activations toward user-given concept values with a proximity-regularised objective, and fine-tune the head by minimising target loss on intervened representations (intervenability), without changing the representations.
Key results
- On synthetic bottleneck data, fine-tuned-for-intervenability black boxes achieve intervention curves comparable to CBMs and improve over the untuned black box.
- On MIMIC-CXR, untuned black boxes are not intervenable, but after intervenability fine-tuning target AUROC rises to 0.808 without interventions and interventions surpass the CBM.
- Fine-tuning for intervenability yields lower Brier scores (better calibration) than the original black box on the synthetic data (0.285 vs 0.460).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune black boxes for intervenability | Black-box classifiers · Synthetic bottleneck | intervention AUROC and AUPR | Helped |
| Fine-tune black boxes for intervenability | Black-box classifiers · Synthetic incomplete | intervention AUROC and AUPR | Helped |
| Use multitask or concept-appending fine-tuning instead of intervenability fine-tuning | Black-box classifiers · Synthetic | intervention AUROC and AUPR | Hurt |
| Use post hoc CBM interventions with incomplete concepts | Post hoc CBM · Synthetic incomplete | intervention performance | Hurt |
| Fine-tune black boxes for intervenability | Black-box classifiers · AwA2 | intervention AUROC and AUPR | Helped |
| Fine-tune black boxes for intervenability | Black-box classifiers · CIFAR-10 | intervention AUROC and AUPR | Helped |
| Fine-tune black boxes for intervenability | Black-box classifiers · MIMIC-CXR | intervention performance | Helped |
| Reduce the validation set size (ablation) | Fine-tuned, I · Synthetic bottleneck | intervention effectiveness | Hurt |
| Increase the intervention tradeoff parameter λ (ablation) | Untuned black box interventions · AwA2 | intervention AUROC and AUPR | Helped |
| Prioritise the most uncertain concepts instead of selecting concepts randomly (ablation) | Untuned black box interventions · AwA2 | intervention performance | Helped |
| Use a nonlinear probe instead of a linear probe (ablation) | Untuned black box interventions · AwA2 | intervention performance | Helped |
| Use residual modelling for post hoc CBM (ablation) | Post hoc CBM · Synthetic incomplete | intervention performance | No clear effect |
Surprising findings
- Post hoc CBMs often see interventions hurt performance on incomplete-concept and chest X-ray settings.
- Fine-tuned black boxes can surpass CBMs on AwA2 and MIMIC-CXR in intervention effectiveness.
Limitations
- Fine-tuning variant keeps representations fixed; the probe is not optimised jointly with the backbone.
- Single fixed intervention strategy used during fine-tuning.
- Effectiveness depends on the probe accurately predicting concepts from activations.
Experimental setup
Datasets: Synthetic tabular (adapted from prior work) AwA2 CUB CIFAR-10 ImageNet MIMIC-CXR CheXpert
Benchmarks: Synthetic tabular AwA2 CIFAR-10 MIMIC-CXR CheXpert ImageNet CUB
Metrics: AUROC AUPR Brier score
Baselines: Black box CBM Post hoc CBM Fine-tuned, MT Fine-tuned, A
Builds on: concept bottleneck models multitask learning with hard parameter sharing counterfactual explanations
Closest papers by meaning
- Energy-Based Concept Bottleneck Models: Unifying Prediction, Concept Intervention, and Probabilistic Interpretations (ICLR 2024 · similarity 0.90)
- Sample-efficient Learning of Concepts with Theoretical Guarantees: from Data to Concepts without Interventions (NeurIPS 2025 · similarity 0.89)
- VLG-CBM: Training Concept Bottleneck Models with Vision-Language Guidance (NeurIPS 2024 · similarity 0.88)
- Concepts Worth Having: Refining VLM-Guided Concept Bottleneck Models with Minimal Annotations (NeurIPS 2026 · similarity 0.87)
- Learning to Intervene on Concept Bottlenecks (ICML 2024 · similarity 0.87)
- Coarse-to-Fine Concept Bottleneck Models (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.