Research map / Continual learning / Class-incremental
Solving the Catastrophic Forgetting Problem in Generalized Category Discovery
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LegoGCD, a plug-in add-on to SimGCD that uses Local Entropy Regularization and a dual-view KL constraint to prevent forgetting of known classes while learning novel classes in generalized category discovery.
Problem
Generalized category discovery suffers from catastrophic forgetting of known-class accuracy when SimGCD's class-mean entropy regularization pushes the model toward novel classes.
Approach
Select confident potential known-class samples in unlabeled data and apply a margin-aware entropy regularizer to them (LER), with a KL term aligning predictions of two augmented views so that the selected samples are more reliable (DKL); both are added to SimGCD without new parameters.
Key results
- On CUB, LegoGCD reaches 72.18% Old accuracy vs 64.44% for SimGCD at end of training (+7.74), and overall All accuracy 63.8% vs 61.9% for SimGCD*.
- On CIFAR100, LegoGCD improves All accuracy to 81.8% from 79.0% (SimGCD*), with Old up 3.5 points.
- On ImageNet-100, All accuracy is 86.3% vs 83.3% for reproduced SimGCD; ImageNet-1k All improves only 0.1 points (62.4 vs 62.3).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add LegoGCD (LER and DKL) to SimGCD | SimGCD · CUB | ACC (Old): 66.5 -> 71.9 | Helped |
| Add LegoGCD (LER and DKL) to SimGCD | SimGCD · Stanford Cars | ACC (All): 53.4 -> 57.3 | Helped |
| Add LegoGCD (LER and DKL) to SimGCD | SimGCD · CIFAR100 | ACC (All): 79.0 -> 81.8 | Helped |
| Add LegoGCD (LER and DKL) to SimGCD | SimGCD · ImageNet-100 | ACC (All): 83.3 -> 86.3 | Helped |
| Add LegoGCD (LER and DKL) to SimGCD | SimGCD · ImageNet-1k | ACC (All): 62.3 -> 62.4 | Helped |
| Add DKL (ablation) | SimGCD · CUB | ACC (All): 61.9 -> 62.5 | Helped |
| Add LER (ablation) | SimGCD · CUB | ACC (Old): 66.5 -> 71.0 | Helped |
| Add LER (ablation) | SimGCD · CUB | ACC (New): 59.6 -> 59.6 | No clear effect |
| Add MAP to LER (ablation) | SimGCD · CUB | ACC (New): 59.6 -> 58.5 | Hurt |
| Add DKL to LER with MAP (ablation) | SimGCD · CUB | ACC (New): 58.5 -> 59.8 | Helped |
| Set β to 2.0 rather than 0.0 (ablation) | LegoGCD · CUB | ACC (Old): 66.5 -> 71.9 | Helped |
| Set β to 2.0 rather than 0.0 (ablation) | LegoGCD · CIFAR100 | ACC (New): 81.5 -> 80.9 | Hurt |
Surprising findings
- Using raw LER alone can reduce accuracy on novel classes, requiring the margin-aware pattern to recover novel-class performance.
- Gains on ImageNet-1k are marginal (+0.1 All, +0.4 Old), and CIFAR10 shows no gain in All accuracy (97.1 vs 96.9).
Limitations
- Relies on a confidence threshold delta that must be tuned per dataset (e.g., 0.85 for Stanford Cars and CIFAR100).
- Gains are small on some datasets (e.g., ImageNet-1k, CIFAR10) and novel-class accuracy can drop with raw LER.
- Requires prior knowledge of total number of categories K, as in prior GCD work.
Experimental setup
Datasets: CIFAR10 CIFAR100 ImageNet-100 CUB Stanford Cars FGVC-Aircraft Herbarium 19 ImageNet-1k
Benchmarks: CIFAR10 CIFAR100 ImageNet-100 CUB Stanford Cars FGVC-Aircraft Herbarium 19 ImageNet-1k
Metrics: clustering accuracy (ACC)
Baselines: k-means RS+ UNO+ ORCA GCD SimGCD
Builds on: SimGCD GCD FixMatch DINO
Closest papers by meaning
- Continual Generalized Category Discovery: Learning and Forgetting from a Bayesian Perspective (ICML 2025 · similarity 0.90)
- DebGCD: Debiased Learning with Distribution Guidance for Generalized Category Discovery (ICLR 2025 · similarity 0.90)
- Active Generalized Category Discovery (CVPR 2024 · similarity 0.89)
- Happy: A Debiased Learning Framework for Continual Generalized Category Discovery (NeurIPS 2024 · similarity 0.89)
- SPTNet: An Efficient Alternative Framework for Generalized Category Discovery with Spatial Prompt Tuning (ICLR 2024 · similarity 0.89)
- Transfer and Alignment Network for Generalized Category Discovery (AAAI 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.