Research map / Continual learning / Class-incremental
Tripartite Weight-Space Ensemble for Few-Shot Class-Incremental Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Tri-WE, which interpolates base, previous, and current classification heads in weight space, plus ADKD distillation on CutMix-amplified few-shot data, to let the full FSCIL model update with few examples.
Problem
Few-shot class-incremental learning suffers catastrophic forgetting and overfitting to scarce new-class examples, and recent methods freeze the feature extractor, limiting adaptability.
Approach
Tri-WE interpolates classification-head weights from the base, previous-session, and current models with learnable scalars while only lightly updating the feature extractor; ADKD distills from the previous model on CutMix-amplified few-shot data to reduce data-bias in distillation.
Key results
- On miniImageNet, Tri-WE with ADKD reaches 70.62% average accuracy and 60.13% last-session accuracy, beating the next best (NC-FSCIL, 67.82% avg) by 1.82 points on average.
- Tri-WE improves average accuracy over the No WE configuration by 2.70 points (70.62% vs 67.92%) on miniImageNet.
- CutMix is the best amplification scheme for ADKD, giving 70.62% avg vs 69.05% without amplification.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Tri-WE with φ0, φold, and φall instead of no weight-space ensemble (ablation) | FSCIL on miniImageNet · miniImageNet | average accuracy (%): 67.92 -> 70.62 | Helped |
| Include the previous-session head φold in the dual-head ensemble (ablation) | FSCIL on miniImageNet · miniImageNet | average accuracy (%): 68.54 -> 69.31 | Helped |
| Apply Tri-WE to classifier head only rather than also ensembling feature-extractor blocks (ablation) | FSCIL on miniImageNet · miniImageNet | average accuracy (%): 65.88 -> 70.62 | Helped |
| Add ADKD to Tri-WE (ablation) | FSCIL on miniImageNet · miniImageNet | average accuracy (%): 69.05 -> 70.62 | Helped |
| Use CutMix rather than CutOut for ADKD data amplification (ablation) | FSCIL on miniImageNet · miniImageNet | average accuracy (%): 69.13 -> 70.62 | Helped |
| Use CutMix rather than RandAug for ADKD data amplification (ablation) | FSCIL on miniImageNet · miniImageNet | average accuracy (%): 69.09 -> 70.62 | Helped |
| Use CutMix rather than MixUp for ADKD data amplification (ablation) | FSCIL on miniImageNet · miniImageNet | average accuracy (%): 70.03 -> 70.62 | Helped |
| Add the feature-level KD loss Lfeat to ADKD (ablation) | FSCIL on miniImageNet · miniImageNet | average accuracy (%): 68.41 -> 70.62 | Helped |
| Add the logit-level KD loss Llogit to ADKD (ablation) | FSCIL on miniImageNet · miniImageNet | average accuracy (%): 70.01 -> 70.62 | Helped |
| Add LCls-Old while updating the feature extractor (ablation) | FSCIL on miniImageNet · miniImageNet | average accuracy (%): 70.15 -> 70.62 | Helped |
| Update the feature extractor rather than freezing it, with LCls-Old (ablation) | FSCIL on miniImageNet · miniImageNet | average accuracy (%): 69.40 -> 70.62 | Helped |
| Replace the strongest reported miniImageNet baseline, NC-FSCIL, with Ours | FSCIL · miniImageNet | average accuracy (%): 67.82 -> 70.62 | Helped |
Surprising findings
- Ensembling only the classification heads is more effective than also ensembling more feature-extractor blocks; increasing the number of ensembled layers degraded performance.
Limitations
- Performance declines as more layers are included in the weight-space ensemble, since merging many layers is hard with few examples.
- Relies on ALICE-style base training plus an auxiliary geometric head; base-session gains depend on these choices.
Experimental setup
Datasets: miniImageNet CUB200 CIFAR100
Benchmarks: miniImageNet CUB200 CIFAR100
Metrics: top-1 accuracy per session average accuracy last session accuracy harmonic mean of base and novel accuracy
Baselines: iCaRL TOPIC CEC ALICE S3C SoftNet NC-FSCIL BiDistill SAVC OrCo CLOSER
Builds on: Wise-FT ALICE NC-FSCIL SAVC CutMix
Closest papers by meaning
- Adaptive Decision Boundary for Few-Shot Class-Incremental Learning (AAAI 2025 · similarity 0.90)
- Pseudo Informative Episode Construction for Few-Shot Class-Incremental Learning (AAAI 2025 · similarity 0.88)
- Pre-trained Vision and Language Transformers Are Few-Shot Incremental Learners (CVPR 2024 · similarity 0.88)
- Diffusion-Classifier Synergy: Reward-Aligned Learning via Mutual Boosting Loop for FSCIL (NeurIPS 2025 · similarity 0.88)
- M2SD:Multiple Mixing Self-Distillation for Few-Shot Class-Incremental Learning (AAAI 2024 · similarity 0.87)
- Compositional Few-Shot Class-Incremental Learning (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.