Research map / Continual learning / Online and replay-based
Incremental Residual Concept Bottleneck Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Res-CBM, a post-hoc method that adds learnable residual concept vectors to a CLIP-based concept bottleneck and converts them into interpretable concepts from a candidate concept bank, plus a Concept Utilization Efficiency metric.
Problem
Concept bottleneck models built from pre-defined CLIP concept banks suffer from incomplete concept coverage, and existing methods lack a measure of how efficiently concepts describe classes.
Approach
Augment the concept bottleneck with optimizable residual vectors that capture missing visual information, then incrementally map each vector to the most similar concept in a candidate concept bank using a concept similarity loss, so the bank stays interpretable while becoming more complete.
Key results
- Res-CBM reaches 0.8803 accuracy on CIFAR-10 with 7 concepts length, beating PCBM-h (0.8757) and Lf-CBM (0.8677) among interpretable methods.
- On CIFAR-100, Res-CBM achieves 0.6791 accuracy with CUE 2.5396, versus Lf-CBM 0.6745 (CUE 0.6301) and LaBo-30c's lower CUE.
- On LAD and CUB, Res-CBM-after averages 0.7009 mean accuracy vs 0.6689 for the supervised Original CBM, though the frozen-backbone LP baseline reaches 0.7714.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply Res-CBM instead of Lf-CBM | CLIP-based CBM · CIFAR-10 | accuracy: 0.8677 -> 0.8803 | Helped |
| apply Res-CBM instead of LP | CLIP-based model · CIFAR-10 | accuracy: 0.8858 -> 0.8803 | Hurt |
| apply Res-CBM instead of Lf-CBM | CLIP-based CBM · CIFAR-100 | accuracy: 0.6745 -> 0.6791 | Helped |
| apply Res-CBM instead of LP | CLIP-based model · CIFAR-100 | accuracy: 0.6995 -> 0.6791 | Hurt |
| add incremental concept discovery to the initial base concept bank (ablation) | Res-CBM · LAD and CUB (Mean) | accuracy: 0.5812 -> 0.7009 | Helped |
| increase the number of residual vectors (ablation) | Lf-CBM on CIFAR-10 · CIFAR-10 | accuracy | Helped |
| vary candidate concept number (ablation) | Res-CBM · CIFAR-10 | accuracy | Mixed |
| vary concept similarity loss weight (ablation) | Res-CBM · CIFAR-10 | accuracy | Mixed |
Surprising findings
- Res-CBM-before (initial base concept bank) performs poorly on LAD and CUB (mean 0.5812), and the gain comes almost entirely from the incremental concept discovery step.
Limitations
- Candidate concept bank construction remains challenging for fine-grained datasets (e.g., Flower-102, CUB-200), where gap to linear probing persists.
- Sequential concept discovery increases computational time cost.
- Existing CBM structure ignores hierarchical composition of concepts.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Tiny-ImageNet LAD CUB-200 Food-101 Flower-102 ConceptNet Broden-style general concept library (Netzer et al.) Visual Genome scene graphs
Benchmarks: CIFAR-10 CIFAR-100 Tiny-ImageNet LAD CUB-200 Food-101 Flower-102
Metrics: accuracy Concept Utilization Efficiency (CUE)
Baselines: CLIP zero-shot Linear probing Original CBM PCBM PCBM-h Lf-CBM LaBo
Builds on: Concept Bottleneck Models Post-hoc CBM (PCBM) Label-free CBM LaBo CLIP
Closest papers by meaning
- Measuring What Matters: Synthetic Benchmarks for Concept Bottleneck Models (NeurIPS 2026 · similarity 0.86)
- Stochastic Concept Bottleneck Models (NeurIPS 2024 · similarity 0.85)
- Editable Concept Bottleneck Models (ICML 2025 · similarity 0.84)
- Hybrid Concept Bottleneck Models (CVPR 2025 · similarity 0.83)
- Energy-Based Concept Bottleneck Models: Unifying Prediction, Concept Intervention, and Probabilistic Interpretations (ICLR 2024 · similarity 0.83)
- A theoretical design of concept sets: improving the predictability of concept bottleneck models (NeurIPS 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.