Research map / Continual learning / Online and replay-based
Quantized Gradient Projection for Memory-Efficient Continual Learning
Dongjun Kim, Seohyeon Cha, Huancheng Chen, Chianing Wang, Haris Vikalo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| quantize GPM to 8 bits | GPM-FP · four CL benchmarks | ACC (%): 71.11 -> 70.70 | No clear effect |
| quantize GPM to 4 bits | GPM-FP · four CL benchmarks | ACC (%): 71.11 -> 69.74 | No clear effect |
| 8QGPM instead of memory-matched rehearsal (DER++-MC) | DER++-MC · 10-split CIFAR100 | ACC (%): 60.63 -> 70.70 | Improved |
| reduce quantization bitwidth from 8 to 4 bits (ablation) | QGPM · 10-split CIFAR100 | ACC (%): 65.01 -> 25.03 | Hurt |
| add outlier handling (2.0% outliers kept in full precision) (ablation) | QGPM · 10-split CIFAR100 | ACC (%): 41.23 -> 55.36 | Improved |
| add quantization-error-aware scaling (alpha=80) to relax orthogonality (ablation) | QGPM · 10-split CIFAR100 | ACC (%): 44.20 -> 67.26 | Improved |
| use large alpha (200) instead of moderate alpha (80) (ablation) | QGPM · 10-split CIFAR100 | ACC (%): 67.26 -> 65.17 | Hurt |
| add sparse sketching (ablation) | 8QGPM · miniImageNet ViT-S | ACC (%): 72.21 -> 72.87 | No clear effect |
| add sparse sketching to reduce GPM construction time (ablation) | 8QGPM · miniImageNet ViT-S | GPM construction time (s): 115.06 -> 10.60 | Improved |
| add sparse sketching to reduce training time (ablation) | 8QGPM · miniImageNet ViT-S | total training time (s): 1768 -> 820 | Improved |
| 8QGPM on ViT trained from scratch | GPM-FP · 10-split miniImageNet (empty ViT) | ACC (%): 48.47 -> 48.39 | No clear effect |
| 8QGPM on NLP text classification | Full precision GPM · Yahoo Answers (5 tasks) | ACC (%): 77.80 -> 77.97 | No clear effect |
Closest papers by meaning
- Continual Learning with Scaled Gradient Projection (AAAI 2023 · similarity 0.86)
- Data Augmented Flatness-aware Gradient Projection for Continual Learning (ICCV 2023 · similarity 0.86)
- An Effective Dynamic Gradient Calibration Method for Continual Learning (ICML 2024 · similarity 0.86)
- TinySubNets: An Efficient and Low Capacity Continual Learning Strategy (AAAI 2025 · similarity 0.85)
- Optimal Rates in Continual Linear Regression via Increasing Regularization (NeurIPS 2025 · similarity 0.85)
- Convergence and Implicit Bias of Gradient Descent on Continual Linear Classification (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
