Research map / Continual learning / Continual fine-tuning of LLMs

Language Guided Concept Bottleneck Models for Interpretable Continual Learning

CVPR 2025 · Continual learning › Continual fine-tuning of LLMs · approach: Continual pretraining of language models · 10 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a language-guided concept bottleneck model framework for exemplar-free class-incremental learning that keeps CLIP-based concepts interpretable while reducing forgetting via concept alignment and semantic-guided prototype augmentation.

Problem

Class-incremental continual learning suffers catastrophic forgetting, and existing CL methods are black boxes whose interpretability degrades as new tasks arrive.

Approach

Builds a continually expanding concept bottleneck layer from ChatGPT-generated, CLIP-selected concepts, aligns concept scores with CLIP concept activations via a similarity loss, and uses semantic-guided prototype augmentation (pseudo-features for old classes from similar new classes) to limit forgetting.

Key results

• Achieves up to 3.06% higher final average accuracy than prior methods on ImageNet-subset B-10 Inc-10 (and at least 1.79% on average incremental accuracy).
• On fine-grained datasets, gains reach 4.43% on CUB-200 and 4.69% on Stanford Cars over the best baselines.
• Against ICICLE with CLIP RN50, reports 66.21 vs 39.31 on CUB-200 B-50 Inc-50 and 69.72 vs 20.85 on Stanford-cars B-20 Inc-20 average incremental accuracy (as laid out in Table 3).

Measured effects reported in the paper

Surprising findings

• Results on CIFAR-100 and Tiny-ImageNet are only comparable to methods that ignore interpretability, not clearly superior.

Limitations

• Interpretability depends on the quality of LLM-generated concepts, which sometimes include non-visual descriptions.
• Concept selection consumes about half of trainable parameters; pre-defined concepts could reduce this but are not explored.

Experimental setup

Datasets: CIFAR-100 Tiny-ImageNet ImageNet-subset CUB-200 Flower Stanford-cars Food-101

Benchmarks: CIFAR-100 Tiny-ImageNet ImageNet-subset CUB-200 Flower Stanford-cars Food-101

Metrics: average incremental accuracy final average accuracy

Baselines: L2P DualPrompt CODA-Prompt CPP LAE Continual-CLIP SLCA EASE ICICLE

Builds on: Concept Bottleneck Models CLIP-based CBM (Yang et al. 2023 label-free CBM) SLCA ICICLE elastic-net sparsity regularization

Closest papers by meaning

• Enhancing Visual Continual Learning with Language-Guided Supervision (CVPR 2024 · similarity 0.90)
• Learning to Intervene on Concept Bottlenecks (ICML 2024 · similarity 0.88)
• A Unified and General Framework for Continual Learning (ICLR 2024 · similarity 0.88)
• Hybrid Concept Bottleneck Models (CVPR 2025 · similarity 0.88)
• Spurious Forgetting in Continual Learning of Language Models (ICLR 2025 · similarity 0.87)
• Stochastic Concept Bottleneck Models (NeurIPS 2024 · similarity 0.87)