Research map / Continual learning / Class-incremental
TIKP: Text-to-Image Knowledge Preservation for Continual Semantic Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TIKP, which builds text prompts from co-occurring seen classes, generates heuristic images with a text-to-image model, and combines a content adaptation loss with a wrong-label knowledge distillation loss to mitigate forgetting in continual semantic segmentation.
Problem
Catastrophic forgetting in continual semantic segmentation (CSS), where a segmentation network forgets previously learned classes when trained on new classes.
Approach
Maintain a compact set of text prompts describing co-occurring seen classes, use a text-to-image model to generate replay-like heuristic images from them, weight each generated image's pseudo-label versus distillation by its feature similarity to old data (content adaption loss), and re-weight distillation per old class by the frequency of wrongly predicted new-class pixels (wrong-label KD).
Key results
- On Pascal VOC 2012 15-1s, TIKP reaches 66.28% mIoU for all classes versus 58.52% for REMINDER and 59.40% for RICL.
- On Pascal VOC 2012 10-1s (11 tasks), TIKP improves all-class mIoU by 21.02 points over the prior best reported.
- On Cityscapes 1-1s, TIKP achieves 50.88% mIoU versus 48.90% for RCIL.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TIKP instead of the strongest reported baseline | Continual semantic segmentation · Pascal VOC 2012, 15-1s | mIoU (all): +6.88% | Helped |
| Use TIKP instead of the strongest reported baseline | Continual semantic segmentation · Pascal VOC 2012, 10-5s | mIoU (all): +5.34% | Helped |
| Use TIKP instead of the strongest reported baseline | Continual semantic segmentation · ADE20k, 50-50s | mIoU (all): 1.06% higher | Helped |
| Use TIKP instead of existing methods | Continual semantic segmentation · Cityscapes | mIoU | Helped |
| Replace Lkd with wrong-label knowledge distillation (ablation) | Baseline segmentation model · Pascal VOC 2012, 15-1s | mIoU (all): 54.68 -> 58.86 | Helped |
| Add Text-to-Image generated images in incremental training (ablation) | Baseline + Lwl-kd · Pascal VOC 2012, 15-1s | mIoU (all): 58.86 -> 63.71 | Helped |
| Add content adaptation loss (ablation) | Baseline + Lwl-kd + TI · Pascal VOC 2012, 15-1s | mIoU (all): 63.71 -> 66.58 | Helped |
Limitations
- The specific text-to-image model is not named in the text.
- Generated images show style inconsistencies with original data, requiring the content adaption loss to compensate.
- Reported gains on some settings are small (e.g., +0.40% on ADE20k 100-50).
Experimental setup
Datasets: Pascal VOC 2012 ADE20k Cityscapes
Benchmarks: Pascal VOC 2012 ADE20k Cityscapes
Metrics: mIoU (old) mIoU (new) mIoU (all) mIoU (avg)
Baselines: ILT MiB SDR PLOP RECALL REMINDER RICL RCIL
Builds on: PLOP MiB CSW-KD RECALL
Closest papers by meaning
- Beyond Background Shift: Rethinking Instance Replay in Continual Semantic Segmentation (CVPR 2025 · similarity 0.90)
- Class Similarity Weighted Knowledge Distillation for Continual Semantic Segmentation (CVPR 2022 · similarity 0.89)
- Enhancing Visual Continual Learning with Language-Guided Supervision (CVPR 2024 · similarity 0.86)
- Continual Vision-Language Retrieval via Dynamic Knowledge Rectification (AAAI 2024 · similarity 0.86)
- AAKR: Adversarial Attack-based Knowledge Retention for Continual Semantic Segmentation (AAAI 2025 · similarity 0.86)
- Towards Continual Universal Segmentation (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.