Research map / Continual learning / Class-incremental
Task-Agnostic Guided Feature Expansion for Class-Incremental Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TagFex, a class-incremental learning framework that continually trains a separate self-supervised task-agnostic model and uses merge attention to transfer its features into expanded task-specific ResNet18 models.
Problem
Feature collision between task-specific features of new and old tasks in expansion-based class-incremental learning, where limited rehearsal samples make auxiliary classifiers imbalanced and suboptimal.
Approach
Continually learn task-agnostic features with a separate contrastive self-supervised model (CaSSLe-style predictive loss), fuse them into the task-specific expanded model via a merge attention, and distill the merged classifier's logits back into the task-specific classifier so inference uses only the task-specific models.
Key results
- On CIFAR100 10-10, TagFex reaches 68.23 last / 78.45 avg accuracy versus DER's 64.35 / 75.36.
- On ImageNet100 50-10, TagFex achieves 70.33 last / 75.87 avg, outperforming DER's 65.27 / 72.60.
- TagFex-P (pruned) cuts averaged incremental parameters from 61.6M to 11.6M on CIFAR100 10-10 with a small accuracy drop (67.34 last).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add task-agnostic features, merge attention, and knowledge transfer to DER | DER · CIFAR100 10-10 | Last accuracy: 64.35 -> 68.23 | Helped |
| add task-agnostic features, merge attention, and knowledge transfer to DER | DER · CIFAR100 50-10 | Last accuracy: 65.27 -> 70.33 | Helped |
| add task-agnostic features, merge attention, and knowledge transfer to DER | DER · ImageNet100 10-10 | Last accuracy: 66.71 -> 70.84 | Helped |
| add task-agnostic features, merge attention, and knowledge transfer to DER | DER · ImageNet100 50-10 | Last accuracy: 71.08 -> 75.54 | Helped |
| add task-agnostic features, merge attention, and knowledge transfer to DER | DER · ImageNet1000 100-100 | Last accuracy: 58.83 -> 61.45 | Helped |
| add continual task-agnostic learning (ablation) | TagFex with task-agnostic model and merge attention but without knowledge transfer · CIFAR100 10-10 | Last accuracy: 64.45 -> 65.86 | Helped |
| add knowledge transfer from merged features to task-specific features (ablation) | TagFex with continual task-agnostic learning · CIFAR100 10-10 | Last accuracy: 65.86 -> 68.23 | Helped |
Surprising findings
- Under a memory-aligned protocol, TagFex with 2000 rehearsal samples outperforms DER given 2300 samples, suggesting the extra task-agnostic model is more memory-efficient than adding rehearsal samples.
Limitations
- Creating a new model per task greatly increases parameter count even though pruning mitigates it.
- Requires storing an extra task-agnostic model across tasks, which adds memory overhead.
Experimental setup
Datasets: CIFAR100 ImageNet100 ImageNet1000
Benchmarks: CIFAR100 ImageNet100 ImageNet1000
Metrics: last-task accuracy average incremental accuracy
Baselines: iCaRL BiC DyTox BEEF DER
Builds on: DER CaSSLe SimCLR FPGM PyCIL
Closest papers by meaning
- Navigating Semantic Drift in Task-Agnostic Class-Incremental Learning (ICML 2025 · similarity 0.87)
- Dynamic Integration of Task-Specific Adapters for Class Incremental Learning (CVPR 2025 · similarity 0.87)
- eTag: Class-Incremental Learning via Embedding Distillation and Task-Oriented Generation (AAAI 2024 · similarity 0.87)
- Task-recency bias strikes back: Adapting covariances in Exemplar-Free Class Incremental Learning (NeurIPS 2024 · similarity 0.86)
- Class Incremental Learning via Likelihood Ratio Based Task Prediction (ICLR 2024 · similarity 0.86)
- Replay-and-Forget-Free Graph Class-Incremental Learning: A Task Profiling and Prompting Approach (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.