Research map / Continual learning / Class-incremental

Data-Free Reservoir Features for Efficient Long-Horizon Cold-Start Continual Learning

NeurIPS 2026 · Continual learning › Class-incremental · approach: Class-balanced knowledge retention · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CIRCLE, a cold-start exemplar-free class-incremental classifier built from fixed untrained BiRC2D-style reservoir features with grouped feature concatenation and prediction averaging over streaming LDA heads.

Problem

Cold-start exemplar-free class-incremental learning, where backbones trained on small first tasks are biased and drift-compensation methods become expensive and unstable over long task horizons.

Approach

Use a never-trained, data-free bidirectional 2D reservoir feature extractor (adapted from BiRC2D) with multiple random draws grouped into feature concatenation and averaged prediction over independent SLDA heads, so the head updates are closed-form and order-invariant with no backbone backpropagation.

Key results

• On CIFAR-100 at T=50, CIRCLE reaches 56.97 mean incremental accuracy and 45.35 final accuracy, versus 40.30 and 26.95 for the best baseline (AdaGauss/EFC++).
• CIRCLE trains fastest on CIFAR-100 at T=20 (2.04 min vs. 9.27 min for the second-fastest method ACIL), about 4.5x faster.
• At T=10, drift-compensation methods such as EFC++ remain stronger than CIRCLE in mean incremental accuracy, so CIRCLE is not universal SOTA at short horizons.

Surprising findings

• A fixed random BiRC2D reservoir without any image training outperforms trained-backbone drift-compensation methods at long horizons (T>=50).
• Random ResNet-18 and VGG-13 features, and scattering transforms, are far weaker than the reservoir extractor under the same SLDA head.

Limitations

• At short horizons (T=10) trained-backbone drift-compensation methods such as EFC++ and AdaGauss outperform CIRCLE.
• Inference requires evaluating multiple reservoir instances and heads, which may add latency in small-batch settings.
• Performance depends on the specific structured reservoir architecture; simpler random extractors perform much worse.

Experimental setup

Datasets: CIFAR-100 TinyImageNet ImageNet-Subset ImageNet-1k

Benchmarks: CIFAR-100 TinyImageNet ImageNet-Subset ImageNet-1k

Metrics: mean incremental accuracy final accuracy wall-clock training time

Baselines: AdaGauss EFC++ ADC ACIL FeCAM FeTRIL IL2A PASS LwF DS-AL

Builds on: BiRC2D SLDA EFC++ AdaGauss RanPAC

Closest papers by meaning

• Elastic Feature Consolidation For Cold Start Exemplar-Free Incremental Learning (ICLR 2024 · similarity 0.88)
• Frozen Feature Augmentation for Few-Shot Image Classification (CVPR 2024 · similarity 0.85)
• Tripartite Weight-Space Ensemble for Few-Shot Class-Incremental Learning (CVPR 2025 · similarity 0.85)
• Advancing Cross-domain Discriminability in Continual Learning of Vision-Language Models (NeurIPS 2024 · similarity 0.85)
• Resurrecting Old Classes with New Data for Exemplar-Free Continual Learning (CVPR 2024 · similarity 0.84)
• REP: Resource-Efficient Prompting for Rehearsal-Free Continual Learning (NeurIPS 2025 · similarity 0.84)