Research map / Continual learning / Class-incremental
Diffusion-Classifier Synergy: Reward-Aligned Learning via Mutual Boosting Loop for FSCIL
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Diffusion-Classifier Synergy (DCS), which uses reward-aligned sampling (DAS with Stable Diffusion 3.5 Medium) guided by feature- and logit-level rewards computed from the FSCIL classifier, in a mutual loop, to generate few images per class for FSCIL.
Problem
Few-shot class-incremental learning suffers from stability-plasticity dilemma and data scarcity; naive diffusion-based data augmentation produces semantically misaligned, low-diversity images and gives no feedback from the classifier.
Approach
A mutual boosting loop in which the classifier's state defines multi-part rewards (prototype-anchored MMD, dimension-wise variance matching, temperature-recalibrated confidence, cross-session confusion-aware) that steer a training-free DAS sampler of a diffusion model, whose generated images then fine-tune the classifier.
Key results
- On mini-ImageNet, DCS reaches 68.14% average accuracy versus 67.05% for the best prior method SAVC (and 66.90% for OrCo).
- On CUB-200, DCS reaches 69.73% average accuracy versus 69.35% for SAVC.
- On CIFAR-100 ablation, the four rewards together raise last-session accuracy by +5.64 points over the baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DCS instead of the strongest listed baseline, SAVC | FSCIL on miniImageNet · miniImageNet | average accuracy (%): 67.05 -> 68.14 | Helped |
| use DCS instead of the strongest listed baseline, SAVC | FSCIL on CUB-200 · CUB-200 | average accuracy (%): 69.35 -> 69.73 | Helped |
| use DCS instead of the strongest listed baseline, ALFSCIL | FSCIL on CIFAR-100 · CIFAR-100 | average accuracy (%): 66.75 -> 66.36 | Hurt |
| add PAMMD reward (ablation) | DCS reward-guided learning · CIFAR-100 | last-session accuracy improvement (%): +1.24 | Helped |
| add VM reward to PAMMD (ablation) | DCS reward-guided learning · CIFAR-100 | last-session accuracy improvement (%): +1.24 -> +1.86 | Helped |
| add RC reward to PAMMD and VM (ablation) | DCS reward-guided learning · CIFAR-100 | last-session accuracy improvement (%): +1.86 -> +3.50 | Helped |
| add CSCA reward to PAMMD, VM, and RC (ablation) | DCS reward-guided learning · CIFAR-100 | last-session accuracy improvement (%): +3.50 -> +5.64 | Helped |
| add PAMMD reward (ablation) | DCS reward-guided learning · miniImageNet | average accuracy (%): 64.03 -> 65.28 | Helped |
| add VM reward after PAMMD (ablation) | DCS reward-guided learning · miniImageNet | average accuracy (%): 65.28 -> 65.57 | Helped |
| add RC reward after PAMMD and VM (ablation) | DCS reward-guided learning · miniImageNet | average accuracy (%): 65.57 -> 66.85 | Helped |
| add CSCA reward after PAMMD, VM, and RC (ablation) | DCS reward-guided learning · miniImageNet | average accuracy (%): 66.85 -> 68.14 | Helped |
| remove any one reward from full DCS (ablation) | DCS on CIFAR-100 · CIFAR-100 | average accuracy (%): 66.36 -> 65.12; 65.54; 64.07; 64.68 | Hurt |
Surprising findings
- The confusion-aware reward degrades FID and CLIP score of generated images, which the authors frame as intentional for generating hard samples.
- Reported CIFAR-100 average accuracy in the main comparison (66.36%) is compared against baselines whose numbers are partly taken from prior papers, and the ablation baseline is not clearly the same as the main results.
Limitations
- Performance depends on access to high-quality pre-trained diffusion models and may drop for domains poorly covered by the diffusion model's training data.
- The multi-component reward and iterative loop add tuning complexity and computational cost.
- Reward design is evaluated only on small benchmarks (miniImageNet, CUB-200, CIFAR-100) with ResNet backbones.
Experimental setup
Datasets: miniImageNet CUB-200 CIFAR-100
Benchmarks: miniImageNet CUB-200 CIFAR-100
Metrics: session accuracy average accuracy FID CLIP score
Baselines: TOPIC CEC FACT TEEN SAVC DyCR ALFSCIL OrCo ADBS
Builds on: Diffusion Alignment as Sampling (DAS) Stable Diffusion 3.5 NCM classifier
Closest papers by meaning
- Tripartite Weight-Space Ensemble for Few-Shot Class-Incremental Learning (CVPR 2025 · similarity 0.88)
- Adaptive Decision Boundary for Few-Shot Class-Incremental Learning (AAAI 2025 · similarity 0.87)
- Diffusion Classifiers Understand Compositionality, but Conditions Apply (NeurIPS 2025 · similarity 0.86)
- Improving Prototypical Visual Explanations with Reward Reweighing, Reselection, and Retraining (ICML 2024 · similarity 0.85)
- Towards Universal Dataset Distillation via Task-Driven Diffusion (CVPR 2025 · similarity 0.85)
- PRDP: Proximal Reward Difference Prediction for Large-Scale Reward Finetuning of Diffusion Models (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.