Research map / Continual learning / Class-incremental
Model Inversion with Layer-Specific Modeling and Alignment for Data-Free Continual Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Per-layer Model Inversion (PMI) with Gaussian/contrastive feature modeling and semantic-aware feature projection to make data-free continual learning via model inversion faster and less prone to synthetic-real feature drift, evaluated on ResNet-32 and CLIP-based CL.
Problem
Data-free class-incremental continual learning needs replay data synthesized by model inversion, but label-driven inversion produces synthetic features that drift from real ones, and inversion is computationally expensive, especially for large ViT/CLIP models.
Approach
Decompose the model inversion KL objective into layer-wise constraints and invert layer by layer (PMI) to get a good initialization for full-model inversion; sample target features from class-wise Gaussians filtered by a contrastively trained MLP (CFS) so synthetic features match real feature distribution; and project old-class image features to new classes via a rotation in CLIP space to synthesi
Key results
- On CIFAR-100 with ResNet-32 (5 tasks), the method reaches 52.38% final accuracy vs 49.87% for R-DFCIL.
- With MoE-Adapter on CIFAR-100, PMI+CFS yields 88.35 average and 81.06 last accuracy vs 87.29 / 79.40 without inversion.
- On CUB-200 with MoE-Adapter, improvement is +5.64 average accuracy (78.98 vs 73.34), while still underperforming rehearsal-based PROOF and CLAP4CLIP on that dataset.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PMI+full-model inversion with CFS instead of R-DFCIL replay | ResNet-32 data-free continual learning · CIFAR-100 and Tiny-ImageNet, 5, 10, and 20 tasks | final average accuracy: +2.51, +2.10, +1.06; +2.57, +3.38, +0.82 | Helped |
| Add synthetic replay using model inversion | VPT, CODA-P, and MoE-Adapter with CLIP · CIFAR-100, ImageNet-R, and CUB-200 | average and last accuracy: +1.47, +2.57; +0.62, +1.30; +1.92, +1.53 (VPT); +1.14, +0.43; +2.66, +2.56; +1.32, +2.80 (CODA-P); +1.06, +1.66; +1.23, | Helped |
| Add PMI initialization to full-model inversion (ablation) | MoE-Adapter with CLIP · CIFAR-100 | Final Acc.: 79.57 -> 81.06 | Helped |
| Add PMI initialization to full-model inversion (ablation) | MoE-Adapter with CLIP · CIFAR-100 | Avg. Acc.: 87.82 -> 88.35 | Helped |
| Use PMI+full-model inversion instead of full-model inversion (ablation) | CLIP-based continual learning · CIFAR-100 | update steps: 3400 -> 600 | Helped |
| Add CFS to Gaussian feature sampling (ablation) | ResNet-32 data-free continual learning · CIFAR-100 and Tiny-ImageNet, 5, 10, and 20 tasks | final average accuracy: +0.33, +0.67, +0.37; +0.25, +0.34, +0.16 | Helped |
| Add CFS to inversion (ablation) | CODA-P+Inversion with CLIP · CIFAR-100 | Avg.: 84.93 -> 84.78 | Hurt |
| Add CFS to inversion (ablation) | CODA-P+Inversion with CLIP · ImageNet-R and CUB-200 | Avg.: 84.32 -> 84.58; 73.32 -> 74.37 | Helped |
| Add CFS to inversion (ablation) | MoE-Adapter+Inversion with CLIP · CIFAR-100, ImageNet-R, and CUB-200 | Avg.: 88.23 -> 88.35; 87.07 -> 87.90; 78.61 -> 78.98 | Helped |
| Add CFS to full-model inversion without PMI (ablation) | MoE-Adapter with CLIP · CIFAR-100 | Avg. Acc.: 87.82 -> 88.13 | Helped |
Surprising findings
- Method underperforms rehearsal-based baselines (PROOF, CLAP4CLIP) on fine-grained CUB-200, though it beats non-rehearsal baselines.
Limitations
- Mapping features to a hypersphere loses scale information, so the distribution is not exactly matched.
- Generating samples for new classes can be improved with LLM class descriptions or more advanced diversity/distribution modeling (e.g., VAEs).
- Underperforms rehearsal-based methods on fine-grained CUB-200.
Experimental setup
Datasets: CIFAR-100 Tiny-ImageNet ImageNet-R CUB-200
Benchmarks: CIFAR-100 Tiny-ImageNet ImageNet-R CUB-200
Metrics: final average accuracy average incremental accuracy last-task accuracy
Baselines: DeepInversion ABD R-DFCIL iCaRL MEMO PROOF CLAP4CLIP ZS-CLIP AttriCLIP VPT CODA-Prompt MoE-Adapter
Builds on: ABD DeepInversion R-DFCIL PROOF MoE-Adapter
Closest papers by meaning
- Patch Rebirth: Fast and Transferable Model Inversion of Vision Transformers (NeurIPS 2026 · similarity 0.87)
- Model Sensitivity Aware Continual Learning (NeurIPS 2024 · similarity 0.87)
- Resurrecting Old Classes with New Data for Exemplar-Free Continual Learning (CVPR 2024 · similarity 0.87)
- MOS: Model Surgery for Pre-Trained Model-Based Class-Incremental Learning (AAAI 2025 · similarity 0.87)
- LoRanPAC: Low-rank Random Features and Pre-trained Models for Bridging Theory and Practice in Continual Learning (ICLR 2025 · similarity 0.87)
- Synthetic Data is an Elegant GIFT for Continual Vision-Language Models (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.