Research map / Continual learning / Class-incremental
OrCo: Towards Better Generalization via Orthogonality and Contrast for Few-Shot Class-Incremental Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OrCo, a three-phase few-shot class-incremental learning framework that pretrains with supervised plus self-supervised contrastive losses, then aligns a frozen backbone's projection head to fixed orthogonal pseudo-targets using a perturbed supervised contrastive loss, cross-entropy, and an orthogonality loss.
Problem
Few-shot class-incremental learning must avoid catastrophic forgetting, overfitting to scarce incremental data, and intransigence (failing to learn new classes) while balancing base pretraining with limited incremental data.
Approach
Combines supervised and self-supervised contrastive pretraining with a frozen backbone and data-independent orthogonal pseudo-targets; a perturbed supervised contrastive loss uses unassigned pseudo-targets as anchors and extra positives to maximize class margins and reserve feature space for future classes.
Key results
- On mini-ImageNet, OrCo reaches aHM 58.12 vs 52.62 for the runner-up NC-FSCIL (+5.5 points).
- On CIFAR100, OrCo improves average harmonic mean by 9.4 points over the runner-up.
- Ablation: integrating PSCL, CE and ORTH yields aHM 58.12; PSCL alone gives 35.74 and CE alone 40.93 on the same ablation table, showing the components are complementary.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use OrCo instead of NC-FSCIL | FSCIL · mini-ImageNet | aHM: 52.62 -> 58.12 | Helped |
| Add ORTH to PSCL + CE (ablation) | OrCo loss · mini-ImageNet | aHM: 36.17 -> 58.12 | Helped |
| Add CE to PSCL + ORTH (ablation) | OrCo loss · mini-ImageNet | aHM: 57.72 -> 58.12 | Helped |
| Add ORTH to CE-only loss (ablation) | OrCo loss · mini-ImageNet | aHM: 40.93 -> 42.73 | Helped |
| Enforce orthogonality when sampling pseudo-targets instead of random sampling (ablation) | OrCo · mini-ImageNet | aHM: 57.23 -> 58.12 | Helped |
| Add pseudo-target perturbations to PSCL (ablation) | OrCo · mini-ImageNet | HM8: 20.14 -> 53.12 | Helped |
| Use uniform rather than Gaussian perturbations in PSCL (ablation) | OrCo · mini-ImageNet | HM8: 50.23 -> 53.12 | Helped |
| Add SSCL to SCL pretraining (ablation) | OrCo pretraining · mini-ImageNet | aHM: 57.38 -> 58.12 | Helped |
| Use SCL + SSCL instead of CE for pretraining (ablation) | OrCo pretraining · mini-ImageNet | aHM: 55.20 -> 58.12 | Helped |
| Fine-tune backbone and projection head instead of only the projection head (ablation) | OrCo incremental training · mini-ImageNet | Performance Decay: 26.99 -> 28.94 | Hurt |
| Fine-tune backbone and projection head instead of only the projection head (ablation) | OrCo incremental training · mini-ImageNet | Base Decay: 17.83 -> 20.52 | Hurt |
Surprising findings
- Plain cross-entropy alignment becomes strongly biased toward base classes, with a 49.26% harmonic-mean drop, while PSCL alone is more stable.
- Applying perturbations to pseudo-targets raises the final-session harmonic mean by over 30 points (20.14 without vs about 53 with perturbations).
Limitations
- Frozen backbone after the pretraining phase limits adaptation to incremental classes.
- On CUB200 the pretraining is skipped and ImageNet-pretrained weights are used instead.
Experimental setup
Datasets: mini-ImageNet CIFAR100 CUB200
Benchmarks: mini-ImageNet CIFAR100 CUB200
Metrics: harmonic mean average harmonic mean (aHM) base accuracy
Baselines: IW FACT CEC C-FSCIL LIMIT LCwoF BiDist NC-FSCIL
Builds on: supervised contrastive learning SimCLR-style self-supervised contrastive learning imprinted weights target generation loss (Hungarian matching) FACT NC-FSCIL
Closest papers by meaning
- Learning With Fantasy: Semantic-Aware Virtual Contrastive Constraint for Few-Shot Class-Incremental Learning (CVPR 2023 · similarity 0.89)
- BECLR: Batch Enhanced Contrastive Few-Shot Learning (ICLR 2024 · similarity 0.88)
- Tripartite Weight-Space Ensemble for Few-Shot Class-Incremental Learning (CVPR 2025 · similarity 0.87)
- Variational Supervised Contrastive Learning (NeurIPS 2025 · similarity 0.86)
- TPR: Topology-Preserving Reservoirs for Generalized Zero-Shot Learning (NeurIPS 2024 · similarity 0.86)
- Pre-trained Vision and Language Transformers Are Few-Shot Incremental Learners (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.