Research map / Continual learning / Class-incremental
Tensor Decomposition Based Memory-Efficient Incremental Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TDMER, which compresses replay exemplars with CP tensor decomposition plus a hybrid herding/random exemplar selection, and plugs it into iCaRL, FOSTER, MEMO and DER to boost class-incremental learning under fixed memory budgets.
Problem
Replay-based class-incremental learning needs many stored exemplars, which is memory-costly; existing pixel-level image compression methods (downsampling, CIM, MECIL) lose discriminative information, especially on low-resolution data.
Approach
Store CP (CANDECOMP/PARAFAC) tensor factors of each image instead of pixels, quantized to 8-bit so the compression ratio is below 1; train with reconstructed images under an adaptive training scheme, and select exemplars with a two-stage hybrid of herding (raw, few) and random (compressed, many) filtered by a relative reconstruction error threshold.
Key results
- On CIFAR-100 (Base0 Inc10, M=2k), TDMER raised iCaRL average accuracy from 60.83% to 67.22% (+6.39) and last accuracy by up to +8.99 points.
- On ImageNet-100 (Base0 Inc10, M=2k), TDMER plus DER reached 79.75% average and 73.26% last accuracy, above MAE-CIL (79.54% / 70.29%).
- On CIFAR-100 compression tests, TDMER (CP rank 16, compression ratio 0.35) reached 69.9% accuracy versus 72.3% for uncompressed training and 66.9% for CIM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| integrate TDMER into iCaRL | iCaRL · CIFAR-100, Base0 Inc10 | last accuracy: 42.99 -> 51.98 | Helped |
| integrate TDMER into DER | DER · CIFAR-100, Base0 Inc20 | last accuracy: 63.17 -> 66.14 | Helped |
| integrate TDMER into DER | DER · ImageNet-100, Base0 Inc10 | last accuracy: 66.84 -> 73.26 | Helped |
| integrate TDMER into DER | DER · ImageNet-100, Base0 Inc20 | average accuracy: 78.56 -> 79.69 | Helped |
| integrate TDMER into MEMO | MEMO · CIFAR-100, Base0 Inc10, M=500 | last accuracy: 45.67 -> 55.76 | Helped |
| integrate TDMER into DER | DER · ImageNet-100, Base0 Inc10, M=500 | last accuracy: 55.86 -> 62.54 | Helped |
| vary CP decomposition rank and exemplar proportion ϵ (ablation) | TDMER integrated into MEMO · CIFAR-100 and ImageNet-100 | average accuracy | Mixed |
| increase CP decomposition rank (ablation) | TDMER integrated into MEMO · ImageNet-100 | average accuracy | Helped |
Surprising findings
- Tucker decomposition needed component sizes close to the original data on low-resolution CIFAR, so the authors used only CP decomposition.
- Some compressed exemplars hurt performance, so emphasizing quantity over quality was counterproductive in memory-efficient CIL.
Limitations
- Gains depend on the exemplar mix hyperparameter epsilon, which the authors tune; ImageNet-100 prefers larger epsilon.
- Tucker decomposition was rejected because it yields components nearly as large as the original data on low-resolution images.
Experimental setup
Datasets: CIFAR-100 ImageNet-100
Benchmarks: CIFAR-100 ImageNet-100
Metrics: average accuracy last accuracy compression ratio
Baselines: iCaRL FOSTER MEMO DER MRDC CIM MAE-CIL MECIL
Builds on: iCaRL FOSTER DER MEMO herding
Closest papers by meaning
- Autoencoder-Based Hybrid Replay for Class-Incremental Learning (ICML 2025 · similarity 0.89)
- Salient Frequency-aware Exemplar Compression for Resource-constrained Online Continual Learning (AAAI 2025 · similarity 0.89)
- DS-AL: A Dual-Stream Analytic Learning for Exemplar-Free Class-Incremental Learning (AAAI 2024 · similarity 0.87)
- Revisiting Generative Replay for Class Incremental Object Detection (CVPR 2025 · similarity 0.87)
- Compress to One Point: Neural Collapse for Pre-Trained Model-Based Class-Incremental Learning (AAAI 2025 · similarity 0.86)
- Distribution-Aware Tensor Decomposition for Compression of Convolutional Neural Networks (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.